Docs/Inference/Chat completions

Chat completions

The OpenAI chat completions shape. Most parameters go straight to the engine untouched.

POST/v1/chat/completions
200
systemYou are terse. No disclaimers.
userWrite the opening line of a heist.
assistant
temperature 0.8max_tokens 256stream false
venym/qwen-27busage 16 in · 0 out
POST/v1/chat/completionskey required
200okapplication/json or text/event-streamBearer vk-…billed per token, in + out
Pick a status to see the error body the gateway returns.

Request

POST /v1/chat/completionscURL
curl https://venym.io/v1/chat/completions \  -H "Authorization: Bearer $VENYM_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "venym/qwen-27b",    "messages": [      {"role": "system", "content": "You are blunt and concise."},      {"role": "user", "content": "Pros and cons of SQLite in production?"}    ],    "temperature": 0.7,    "max_tokens": 600  }'

Required

modelstring
A model id from /v1/models. Add :online for web search. Unknown ids return 404 model_not_found.
messagesarray
Non-empty list of {role, content} objects. Text content may be a string or an array of parts. Empty or missing returns 400.

Passed through to the engine

Everything else in the body is forwarded as-is to the model server (an OpenAI-compatible engine), so any parameter the engine understands works. The ones you will use:

temperaturenumber
Sampling temperature.
top_pnumber
Nucleus sampling.
max_tokensinteger
Max completion tokens. Capped at the model's max output (see the table on models). If omitted, the cap itself is used. max_completion_tokens is read the same way.
stopstring | array
Stop sequences.
seedinteger
Best-effort determinism. Depends on the engine.
presence_penaltynumber
Passed through.
frequency_penaltynumber
Passed through.
tools, tool_choicearray, string | object
Function calling. See tools.
response_formatobject
For example {"type": "json_object"}. Both models list the json capability.
streamboolean
Server-sent events. See streaming.
stream_optionsobject
When stream is true this is replaced with {"include_usage": true}, always.

Venym extras

One optional top-level object, stripped before the request reaches the engine:

venym.web_searchboolean
Same as the :online suffix. See web search.
venym.search_resultsinteger
How many results to ground on. Default 6, max 10.
payload.jsonJSON
{  "model": "venym/glm-flash",  "messages": [{"role": "user", "content": "What shipped in the latest Postgres release?"}],  "venym": { "web_search": true, "search_results": 8 }}

Response

A standard chat.completion object. The model field is always the Venym id you asked for, not the name of the server behind it.

response.jsonJSON
{  "id": "chatcmpl-...",  "object": "chat.completion",  "created": 1767225600,  "model": "venym/qwen-27b",  "choices": [    {      "index": 0,      "message": { "role": "assistant", "content": "..." },      "finish_reason": "stop"    }  ],  "usage": { "prompt_tokens": 42, "completion_tokens": 180, "total_tokens": 222 },  "venym": {    "sources": [      { "title": "Example page", "url": "https://example.com/post" }    ]  }}
  • usage is what you are billed on. If the engine does not report it, we estimate from text length (about four characters per token).
  • venym.sources appears only when web search ran and returned results. It is a list of {title, url}, in the same order as the [n] citations in the text.

Behavior worth knowing

  • The balance check happens before the call, billing after it. A request that starts with a positive balance can finish with a slightly negative one.
  • If one inference node is down or returns a 5xx or 429, the gateway tries another before giving you an error.
  • 4xx errors from the engine (a bad parameter, say) are returned to you with the engine's message and are not retried.

Errors use the OpenAI shape. See errors.