Chat Completions
The chat completions endpoint generates a response from a model or an Assistant. It follows the OpenAI Chat Completions API, so you can use it with the OpenAI libraries and with any framework that supports them.
Endpoint
POST /v1/chat/completions
The endpoint requires the chat.completions scope. See Authentication.
Request Body
| Field | Type | Required | Description |
|---|---|---|---|
model |
string | Yes | What should answer: a model ID, a model name, or an Assistant such as assistant:hr-helper. See Listing Models |
messages |
array | Yes | The conversation. Each item has a role (system, user, assistant or tool) and content |
stream |
boolean | No | If true, the response is streamed as Server-Sent Events. Default false |
temperature |
number | No | Randomness, 0–2. Higher is more varied |
top_p |
number | No | Nucleus sampling threshold |
max_completion_tokens |
integer | No | The most tokens the response may contain. max_tokens is accepted too |
stop |
string or array | No | Sequences where generation stops |
reasoning_effort |
string | No | How much a reasoning model may think first: minimal, low, medium or high. See Reasoning |
response_format |
object | No | {"type": "json_object"} or a json_schema for structured output |
tools |
array | No | Tool definitions the model may call. See Tools |
tool_choice |
string or object | No | "auto", "none", "required", or a specific tool |
frequency_penalty |
number | No | Reduces repetition of frequent tokens, -2 to 2 |
presence_penalty |
number | No | Reduces repetition of any earlier token, -2 to 2 |
seed |
integer | No | Makes sampling repeatable |
Settings you leave out will use the model's own defaults, set by your administrator.
Basic Example
Request:
{
"model": "fp:9189ab12cd34ef56",
"messages": [
{ "role": "system", "content": "You are a helpful assistant." },
{ "role": "user", "content": "What is the capital of France?" }
]
}
Response:
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1718000000,
"model": "Qwen3.5-9B",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The capital of France is Paris."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 28,
"completion_tokens": 9,
"total_tokens": 37
}
}
Streaming
You can set "stream": true to receive the response as Server-Sent Events. Each event will carry a part of the response.
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant","content":"Why"},"index":0}]}
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"content":" don't"},"index":0}]}
...
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{},"finish_reason":"stop","index":0}]}
data: [DONE]
Note
With response_format set to json_object, the response will not be streamed, even if stream is true.
Using An Assistant
You can send a request to an Assistant instead of a model. Set model to an Assistant's reference, such as assistant:hr-helper. The Assistant will answer with its own model, instructions, tools and Knowledge Bases. The Assistant must be available on the API, and shared with your key or user.
For example:
{
"model": "assistant:hr-helper",
"messages": [{ "role": "user", "content": "How many days of annual leave do I get?" }]
}
- The Assistant's temperature and reasoning effort, if set, take precedence over the request's.
- What happens to a
systemmessage you send depends on the Assistant's settings: it may be added to the Assistant's instructions, ignored, used instead, or refused. See Assistants.
| Error | Status | Meaning |
|---|---|---|
model_not_found |
404 | No such Assistant or model — or it is not available to you, or not on the API |
model_not_available |
503 | The Assistant exists, but none of its models are running |
system_prompt_not_allowed |
400 | The Assistant refuses requests that bring their own system prompt |
Images
Models that can read images accept them as message content parts. The image must be inline, as a data: URI. For example:
{
"model": "fp:9189ab12cd34ef56",
"messages": [{
"role": "user",
"content": [
{ "type": "text", "text": "What does this error message say?" },
{ "type": "image_url", "image_url": { "url": "data:image/png;base64,iVBORw0KGgo..." } }
]
}]
}
Check a model's input_modalities in Listing Models to see whether it reads images. Requests will be refused with 400 and the code unsupported_media when:
- an image is given as a web link — send it inline instead
- the model is known not to read images
Reasoning
For models that think before answering, you can use reasoning_effort to set how much they think:
| Value | Effect |
|---|---|
| (not sent) | The model's own setting |
minimal |
No thinking — answers straight away |
low |
Thinking capped at about 1,000 tokens |
medium |
Thinking capped at about 4,000 tokens |
high |
Thinks as long as it needs |
A capped model answers sooner, but hard questions are more likely to come out wrong.
Tools
A request can use two kinds of tool: tools the Gateway runs for you, and your own tools, which you run yourself.
Tools The Gateway Runs
Your administrator may make tools available on the API — Connectors, MCP servers, Knowledge Base search, or the tools of an Assistant you call. When there are any, OnVeris runs the tool loop itself: the model calls a tool, the Gateway runs it, gives the model the result, and repeats until the model answers — or calls one of your own tools. You will receive only the final answer.
When streaming, the response will then also carry named events showing the tool activity, alongside the usual data: chunks. For example:
event: agent_tool_start
data: {"call_id":"call_1","tool_name":"search_knowledge_base","args":"{\"query\":\"annual leave\"}"}
event: agent_tool_result
data: {"call_id":"call_1","result":"…","duration_ms":412,"error":null}
| Event | When |
|---|---|
agent_thinking_start |
A round of the loop begins |
agent_tool_start |
A tool is called |
agent_tool_result |
A tool returns |
agent_note |
Something the caller should know — for example earlier tool results were dropped to fit the context window, or the loop hit its limit |
agent_loop_end |
The loop has finished |
agent_done |
The response is complete |
agent_error |
The loop failed |
Clients that do not understand these events can ignore them: the answer will arrive in the usual data: chunks.
The number of tool rounds is limited by the Gateway's settings. Tools that must ask a person's permission are never offered through the API, since there is nobody to ask.
When Knowledge Base passages were added to the request automatically, the response will carry the header X-OnVeris-Knowledge-Injected with the number of passages. It will also carry X-Optima-Knowledge-Injected, the same header under its name before the rename, for clients that read that one.
Your Own Tools
Tools you send in tools work as they do with OpenAI: you run them, not the Gateway. They are offered to the model alongside any tools the Gateway has. When the model calls one of yours, the response will end with tool_calls instead of text. For example:
{
"choices": [
{
"message": {
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_xyz",
"type": "function",
"function": {
"name": "get_weather",
"arguments": "{\"city\":\"London\"}"
}
}
]
},
"finish_reason": "tool_calls"
}
]
}
When streaming, each call will arrive as a tool_calls delta, followed by a final chunk with "finish_reason": "tool_calls" and data: [DONE].
Run the function yourself, add the model's message and a tool message with the result to messages, and call the endpoint again for the next step.
- If the model calls Gateway tools and yours in the same step, the Gateway will run its own first, then return only the calls to yours.
- The Gateway's results are not part of your conversation. When you come back with your tool results, the model may call one of the Gateway's tools again.
- When you call an Assistant, your tools are only offered if the Assistant allows API callers' own tools. Otherwise they will be left out. See Assistants.
Controlling The Gateway's Tool Loop
You can control whether the Gateway runs its tools for a request with a header or with tool_choice:
| Header or field | Effect |
|---|---|
X-OnVeris-Agent: false |
Do not use the Gateway's tools for this request. Your own tools are still offered |
X-OnVeris-Agent: true |
Use them |
"tool_choice": "none" |
Do not use tools at all |
"tool_choice": "required" |
Use the Gateway's tools |
A Gateway also reads X-Optima-Agent, the header's name before OnVeris was renamed.
Conversations
/v1/chat/completions keeps no history between requests. Send the whole conversation in messages each time. To have the Gateway keep the history for you, use Responses.
For example:
{
"model": "fp:9189ab12cd34ef56",
"messages": [
{ "role": "user", "content": "My name is Alice." },
{ "role": "assistant", "content": "Hello Alice! How can I help you today?" },
{ "role": "user", "content": "What is my name?" }
]
}
Context Length
If the messages, plus room for the answer, do not fit the model's context window, the request will be refused rather than shortened:
{
"error": {
"message": "…",
"type": "context_length_exceeded",
"param": "messages",
"code": "context_length_exceeded"
},
"promptTokens": 40210,
"contextWindow": 32768
}
Shorten the conversation and try again. Use Listing Models to see each model's ctx_max.
See Errors and Limits for other errors.