APIs

Responses API

The newer /v1/responses endpoint is fully supported, including the AI Gateway-specific features: hosted Code Interpreter (via the caronte:code_execution tool) and Prompt Vault references through OpenAI's native prompt param.

Basic usage

Pass instructions (system message) and input (user turn). Read output_text for the answer.

example.ts

import { client } from "./client";

const response = await client.responses.create({
  model: "gpt-4o-mini",
  instructions: "You are a concise assistant.",
  input: "Summarize what AI Gateway does in one sentence.",
});

console.log(response.output_text);

Streaming

Set stream: true for a richer typed event sequence (response.output_text.delta, response.completed, …) — the SDK exposes them as typed events.

example.ts

import { client } from "./client";

const stream = await client.responses.create({
  model: "gpt-4o-mini",
  input: "Stream a short haiku about proxies.",
  stream: true,
});

for await (const event of stream) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta);
  }
}

Streaming is rejected on keys with active PII masking (400) — masking can't be applied mid-stream.

Structured input

input also accepts a list of items — messages, function_call_outputs, file inputs, etc.

example.ts

import { client } from "./client";

const response = await client.responses.create({
  model: "gpt-4o-mini",
  input: [
    { role: "system", content: "You answer in JSON." },
    {
      role: "user",
      content: [
        { type: "input_text", text: "Extract name and city from: 'Anna, Milan'" },
      ],
    },
  ],
});

console.log(response.output_text);

Stateless conversations

The gateway serves the Responses API stateless-only: every call is forced to store: false upstream, and when the request speaks reasoning (a reasoning param, or reasoning items replayed in input) the gateway also asks for the reasoning state inline as encrypted content (adding include: ["reasoning.encrypted_content"] automatically; deployments that don't support it simply omit reasoning items — the replay contract is identical either way). This is what lets a model name load-balance across multiple provider deployments — stored item ids are bound to the specific provider resource that created them, so nothing may depend on server-side storage.

example.ts

import { client } from "./client";

// Multi-turn: replay the previous output items (reasoning included,
// unmodified) followed by the new turn — no previous_response_id.
const first = await client.responses.create({
  model: "gpt-4o-mini",
  input: "Pick a number between 1 and 10.",
  store: false, // explicit acknowledgement — silences the Warning header
});

const followUp = await client.responses.create({
  model: "gpt-4o-mini",
  input: [
    ...first.output,
    { role: "user", content: "Double it." },
  ],
  store: false,
});

warning · previous_response_id is rejected

A request carrying previous_response_id fails with 400 — there is no stored previous response to reference. Replay the full conversation in input instead (the OpenAI Agents SDK already does this). Requests that omit store or set it to true still succeed, but carry a Warning response header (299) noting that store: false was applied. response.id values cannot be retrieved later — persist what you need client-side. Prompt caching is unaffected.

Identifying the end user

Like Chat Completions, /v1/responses accepts the standard OpenAI user field (or the X-User header) to attribute the generation, enforce per-user budgets/caps, and satisfy an API key's force_user requirement.

example.ts

import { client } from "./client";

await client.responses.create({
  model: "gpt-4o-mini",
  input: "Hello",
  user: "user_42",
});

note · force_user & per-user limits

When the API key has force_user enabled, a request without a user (or the X-User header) is rejected with 400. The identifier also selects which per-user budget or request cap applies. Full behavior in End-user identifier.

Prompt Vault

Reference a stored prompt through OpenAI's native prompt param instead of hard-coding it. Drop input/instructions and the prompt provides them; keep input and the prompt's system part is merged into instructions.

example.ts

import { client } from "./client";

const response = await client.responses.create({
  model: "gpt-4o-mini",
  // Keep your own input; the vault prompt's system part is merged
  // into instructions (vault-first — your instructions are kept).
  input: "App keeps crashing on launch",
  prompt: { id: "support/triage", version: "production" },
} as never);

tip · Merge semantics

No input → the compiled prompt becomes the conversation; with input → only the prompt's system part is used, prepended to any instructions you sent (vault-first). Full cross-endpoint rules in Prompt Vault → Merge.