Skip to content

Responses

The responses endpoint is a stateful alternative to /v1/chat/completions. The Gateway stores each turn on the server, so clients only need to send the new user message on later requests. The earlier conversation history will be reconstructed automatically from previous_response_id.

Endpoint

POST /v1/responses

The endpoint requires the responses scope. See Authentication.

Request Body

Field Type Required Description
model string Yes What should answer: a model ID, a model name, or an Assistant such as assistant:hr-helper. See Listing Models.
input string or array Yes The new user message as a string, or an array of items — messages, and function_call_output items answering tool calls. See Tools.
previous_response_id string No The id of the previous response. When provided, the Gateway prepends the full conversation history automatically.
instructions string No A system prompt. On the first turn this is stored alongside the conversation. On subsequent turns it can be omitted and the original instructions will be reused.
tools array No Function tools the model may call. See Tools.
tool_choice string or object No "auto", "none", "required", or a specific tool object.
stream boolean No If true, the response is streamed using the event format described below. Default: false.
store boolean No If false, the response is not persisted and previous_response_id chaining is disabled for this turn. Default: true.
temperature number No Sampling temperature.
max_output_tokens integer No Maximum tokens in the response.
response_format object No {"type": "json_object"} or a json_schema for structured output.

Basic Example

In this example we will ask a question, then continue the conversation with a second one.

First turn (no previous_response_id):

{
  "model": "llama-3.1-8b",
  "input": "What is the capital of France?",
  "instructions": "You are a helpful assistant."
}

Response:

{
  "id": "resp_a1b2c3d4e5f6...",
  "object": "response",
  "created_at": 1718000000,
  "model": "llama-3.1-8b",
  "status": "completed",
  "output": [
    {
      "type": "message",
      "id": "abc123",
      "role": "assistant",
      "content": [
        { "type": "output_text", "text": "The capital of France is Paris." }
      ]
    }
  ],
  "usage": {
    "input_tokens": 28,
    "output_tokens": 9,
    "total_tokens": 37
  }
}

Second turn (continuing the conversation):

{
  "model": "llama-3.1-8b",
  "input": "And what about Germany?",
  "previous_response_id": "resp_a1b2c3d4e5f6..."
}

The Gateway will look up the earlier response, reconstruct the conversation history, and return a response as if the full history had been sent.

Response Object

Field Description
id Unique response identifier (resp_...). Pass this as previous_response_id on the next turn.
object Always "response".
created_at Unix timestamp.
model The model that generated the response.
status Always "completed" for non-streaming responses.
output Array of output items. Each item has a type of "message" (text) or "function_call" (a call to one of your tools).
usage.input_tokens Prompt token count.
usage.output_tokens Completion token count.
usage.total_tokens Combined token count.

Tools

You can give the model function tools of your own to call. Send tool definitions in tools, in the Responses API's format or the Chat Completions format — both are read. For example:

{
  "type": "function",
  "name": "get_weather",
  "description": "Get the current weather for a city",
  "parameters": {
    "type": "object",
    "properties": { "city": { "type": "string" } },
    "required": ["city"]
  }
}

Only function tools are supported. OpenAI's hosted tools, such as web_search or file_search, will be refused with 400 and unsupported_tool_type.

When The Model Calls One Of Your Tools

The output array will contain a function_call item, after any text the model wrote. For example:

{
  "output": [
    {
      "type": "function_call",
      "call_id": "call_xyz",
      "name": "get_weather",
      "arguments": "{\"city\":\"London\"}"
    }
  ]
}

When streaming, it will arrive as response.output_item.added, response.function_call_arguments.done and response.output_item.done events.

Run the function, then send the result as a function_call_output item in input, with the response's id as previous_response_id:

{
  "model": "fp:9189ab12cd34ef56",
  "previous_response_id": "resp_a1b2c3d4e5f6...",
  "input": [
    { "type": "function_call_output", "call_id": "call_xyz", "output": "{\"temp_c\": 14}" }
  ]
}

A function_call_output whose call_id does not match a call in the conversation will be refused with 400.

Tools The Gateway Runs

As with chat completions, the Gateway may have tools of its own for a request — Connectors, MCP servers, Knowledge Base search, or an Assistant's tools. It runs those itself, and they are offered to the model alongside yours. If the model calls Gateway tools and yours in the same step, the Gateway will run its own first and return only the calls to yours. The Gateway's tool activity is not shown in the response.

What The Input Can Contain

input can be a string, or an array of items: messages (with role user, assistant, system or developer), function_call items and function_call_output items. Message content must be text. Anything else will be refused with 400, naming what was not understood:

Code Meaning
unsupported_content_type A message part that is not text, such as an image
unsupported_input_item An input item of a type OnVeris does not read
unsupported_tool_type A tool that is not a function tool

Streaming

You can set "stream": true to receive the response as Server-Sent Events. For example:

event: response.created
data: {"type":"response.created","response":{"id":"resp_...","status":"in_progress",...}}

event: response.output_item.added
data: {"type":"response.output_item.added","output_index":0,"item":{"type":"message","role":"assistant","content":[]}}

event: response.output_text.delta
data: {"type":"response.output_text.delta","output_index":0,"content_index":0,"delta":"The capital"}

event: response.output_text.delta
data: {"type":"response.output_text.delta","output_index":0,"content_index":0,"delta":" of France is Paris."}

event: response.output_text.done
data: {"type":"response.output_text.done","output_index":0,"content_index":0,"text":"The capital of France is Paris."}

event: response.output_item.done
data: {"type":"response.output_item.done","output_index":0,"item":{...}}

event: response.completed
data: {"type":"response.completed","response":{...}}

data: [DONE]

If generation fails part-way, a response.error event will be sent instead of response.completed.

Opting Out Of Persistence

You can set "store": false to process the request without saving it. The response will not be retrievable through previous_response_id, and no conversation record will be created.

Long Conversations

The Gateway keeps every turn, but a model can only read so much at once. When the stored history no longer fits the model's context window, the oldest turns will be left out of what the model reads, so the conversation can carry on. Only if your new input is too large on its own will the request be refused, with 400 and context_length_exceeded.

Who Can Continue A Conversation

A conversation belongs to whoever started it: the API Key that made the request, or the signed-in user for a request made with an access token. Only its owner can continue it — a user cannot continue a conversation started with one of their API Keys, or the other way round.

If previous_response_id names a response that does not exist, was made with "store": false, or belongs to someone else, the request will be refused with 404:

{
  "error": {
    "message": "Previous response with id 'resp_…' not found.",
    "type": "invalid_request_error",
    "param": "previous_response_id",
    "code": "previous_response_not_found"
  }
}

There is no endpoint to fetch or delete a stored response.