Responses
The responses endpoint is a stateful alternative to /v1/chat/completions. The Gateway stores each turn on the server, so clients only need to send the new user message on later requests. The earlier conversation history will be reconstructed automatically from previous_response_id.
Endpoint
POST /v1/responses
The endpoint requires the responses scope. See Authentication.
Request Body
| Field | Type | Required | Description |
|---|---|---|---|
model |
string | Yes | What should answer: a model ID, a model name, or an Assistant such as assistant:hr-helper. See Listing Models. |
input |
string or array | Yes | The new user message as a string, or an array of items — messages, and function_call_output items answering tool calls. See Tools. |
previous_response_id |
string | No | The id of the previous response. When provided, the Gateway prepends the full conversation history automatically. |
instructions |
string | No | A system prompt. On the first turn this is stored alongside the conversation. On subsequent turns it can be omitted and the original instructions will be reused. |
tools |
array | No | Function tools the model may call. See Tools. |
tool_choice |
string or object | No | "auto", "none", "required", or a specific tool object. |
stream |
boolean | No | If true, the response is streamed using the event format described below. Default: false. |
store |
boolean | No | If false, the response is not persisted and previous_response_id chaining is disabled for this turn. Default: true. |
temperature |
number | No | Sampling temperature. |
max_output_tokens |
integer | No | Maximum tokens in the response. |
response_format |
object | No | {"type": "json_object"} or a json_schema for structured output. |
Basic Example
In this example we will ask a question, then continue the conversation with a second one.
First turn (no previous_response_id):
{
"model": "llama-3.1-8b",
"input": "What is the capital of France?",
"instructions": "You are a helpful assistant."
}
Response:
{
"id": "resp_a1b2c3d4e5f6...",
"object": "response",
"created_at": 1718000000,
"model": "llama-3.1-8b",
"status": "completed",
"output": [
{
"type": "message",
"id": "abc123",
"role": "assistant",
"content": [
{ "type": "output_text", "text": "The capital of France is Paris." }
]
}
],
"usage": {
"input_tokens": 28,
"output_tokens": 9,
"total_tokens": 37
}
}
Second turn (continuing the conversation):
{
"model": "llama-3.1-8b",
"input": "And what about Germany?",
"previous_response_id": "resp_a1b2c3d4e5f6..."
}
The Gateway will look up the earlier response, reconstruct the conversation history, and return a response as if the full history had been sent.
Response Object
| Field | Description |
|---|---|
id |
Unique response identifier (resp_...). Pass this as previous_response_id on the next turn. |
object |
Always "response". |
created_at |
Unix timestamp. |
model |
The model that generated the response. |
status |
Always "completed" for non-streaming responses. |
output |
Array of output items. Each item has a type of "message" (text) or "function_call" (a call to one of your tools). |
usage.input_tokens |
Prompt token count. |
usage.output_tokens |
Completion token count. |
usage.total_tokens |
Combined token count. |
Tools
You can give the model function tools of your own to call. Send tool definitions in tools, in the Responses API's format or the Chat Completions format — both are read. For example:
{
"type": "function",
"name": "get_weather",
"description": "Get the current weather for a city",
"parameters": {
"type": "object",
"properties": { "city": { "type": "string" } },
"required": ["city"]
}
}
Only function tools are supported. OpenAI's hosted tools, such as web_search or file_search, will be refused with 400 and unsupported_tool_type.
When The Model Calls One Of Your Tools
The output array will contain a function_call item, after any text the model wrote. For example:
{
"output": [
{
"type": "function_call",
"call_id": "call_xyz",
"name": "get_weather",
"arguments": "{\"city\":\"London\"}"
}
]
}
When streaming, it will arrive as response.output_item.added, response.function_call_arguments.done and response.output_item.done events.
Run the function, then send the result as a function_call_output item in input, with the response's id as previous_response_id:
{
"model": "fp:9189ab12cd34ef56",
"previous_response_id": "resp_a1b2c3d4e5f6...",
"input": [
{ "type": "function_call_output", "call_id": "call_xyz", "output": "{\"temp_c\": 14}" }
]
}
A function_call_output whose call_id does not match a call in the conversation will be refused with 400.
Tools The Gateway Runs
As with chat completions, the Gateway may have tools of its own for a request — Connectors, MCP servers, Knowledge Base search, or an Assistant's tools. It runs those itself, and they are offered to the model alongside yours. If the model calls Gateway tools and yours in the same step, the Gateway will run its own first and return only the calls to yours. The Gateway's tool activity is not shown in the response.
What The Input Can Contain
input can be a string, or an array of items: messages (with role user, assistant, system or developer), function_call items and function_call_output items. Message content must be text. Anything else will be refused with 400, naming what was not understood:
| Code | Meaning |
|---|---|
unsupported_content_type |
A message part that is not text, such as an image |
unsupported_input_item |
An input item of a type OnVeris does not read |
unsupported_tool_type |
A tool that is not a function tool |
Streaming
You can set "stream": true to receive the response as Server-Sent Events. For example:
event: response.created
data: {"type":"response.created","response":{"id":"resp_...","status":"in_progress",...}}
event: response.output_item.added
data: {"type":"response.output_item.added","output_index":0,"item":{"type":"message","role":"assistant","content":[]}}
event: response.output_text.delta
data: {"type":"response.output_text.delta","output_index":0,"content_index":0,"delta":"The capital"}
event: response.output_text.delta
data: {"type":"response.output_text.delta","output_index":0,"content_index":0,"delta":" of France is Paris."}
event: response.output_text.done
data: {"type":"response.output_text.done","output_index":0,"content_index":0,"text":"The capital of France is Paris."}
event: response.output_item.done
data: {"type":"response.output_item.done","output_index":0,"item":{...}}
event: response.completed
data: {"type":"response.completed","response":{...}}
data: [DONE]
If generation fails part-way, a response.error event will be sent instead of response.completed.
Opting Out Of Persistence
You can set "store": false to process the request without saving it. The response will not be retrievable through previous_response_id, and no conversation record will be created.
Long Conversations
The Gateway keeps every turn, but a model can only read so much at once. When the stored history no longer fits the model's context window, the oldest turns will be left out of what the model reads, so the conversation can carry on. Only if your new input is too large on its own will the request be refused, with 400 and context_length_exceeded.
Who Can Continue A Conversation
A conversation belongs to whoever started it: the API Key that made the request, or the signed-in user for a request made with an access token. Only its owner can continue it — a user cannot continue a conversation started with one of their API Keys, or the other way round.
If previous_response_id names a response that does not exist, was made with "store": false, or belongs to someone else, the request will be refused with 404:
{
"error": {
"message": "Previous response with id 'resp_…' not found.",
"type": "invalid_request_error",
"param": "previous_response_id",
"code": "previous_response_not_found"
}
}
There is no endpoint to fetch or delete a stored response.