Docs/Inference/Tools

Tools

Function calling, OpenAI style. Both models carry the tools capability.

POSTtools: [get_weather]
1 / 4
assistantfinish: tool_calls
{ "role": "assistant", "content": null,
"tool_calls": [{ "id": "call_1", "type": "function",
"function": { "name": "get_weather",
"arguments": "{\"city\":\"Oslo\"}" } }] }
messages[] so far2 items
0userWhat is the weather in Oslo?
1assistanttool_calls: get_weather({"city":"Oslo"})
2toolnot yet
3assistantnot yet

How it works

Send tools (and optionally tool_choice) in the request. They are passed to the engine unchanged. When the model wants a function run, the response has finish_reason: "tool_calls" and a tool_calls array on the message. You run the function, append the result as a tool message, and call again.

Example

import json, osfrom openai import OpenAI client = OpenAI(base_url="https://venym.io/v1", api_key=os.environ["VENYM_API_KEY"]) tools = [{    "type": "function",    "function": {        "name": "get_weather",        "description": "Current weather for a city.",        "parameters": {            "type": "object",            "properties": {"city": {"type": "string"}},            "required": ["city"],        },    },}] def get_weather(city):    return {"city": city, "temp_c": 11, "sky": "rain"}  # your real lookup here messages = [{"role": "user", "content": "Do I need an umbrella in Bristol?"}]MODEL = "venym/glm-flash"  # or venym/qwen-27b first = client.chat.completions.create(model=MODEL, messages=messages, tools=tools, tool_choice="auto")msg = first.choices[0].message if msg.tool_calls:    messages.append(msg)    for call in msg.tool_calls:        args = json.loads(call.function.arguments)        messages.append({            "role": "tool",            "tool_call_id": call.id,            "content": json.dumps(get_weather(**args)),        })    final = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)    print(final.choices[0].message.content)else:    print(msg.content)

tool_choice

  • "auto": the model decides. The default.
  • "none": never call a tool.
  • "required" or {"type": "function", "function": {"name": "..."}}: force a call. Support for forcing depends on the engine behind the model; if one form misbehaves, try the other.

Tips

  • Tool arguments arrive as a JSON string. Parse defensively: models sometimes produce slightly invalid JSON, especially under high temperature. Lower the temperature for tool-heavy agents.
  • Tools work with streaming. Arguments arrive as fragments in delta.tool_calls; concatenate by index before parsing.
  • Tool definitions and results count as prompt tokens. Keep schemas short.
  • Tools combine with :online. The search runs once up front, from the last user message; the model can still call your functions after.