AI Gateway features

Prompt Vault

Reference a prompt stored in the Prompt Vault instead of hard-coding it. The proxy resolves tag/version to a commit, substitutes variables, and merges the result into the request before guardrails and PII masking run. Available on /v1/responses (OpenAI's native prompt param), /v1/chat/completions and /anthropic/v1/messages (same object as an extra-body field). Not supported on the token-counting endpoints.

Invoke a stored prompt

example.ts

import { client } from "./client";

const response = await client.responses.create({
  model: "gpt-4o-mini",
  // No input/instructions: the Prompt Vault entry provides them.
  prompt: {
    id: "support/triage",          // project/prompt path
    version: "production",          // tag or commit id (omit for latest)
    variables: { ticket_subject: "App keeps crashing on launch" },
  },
} as never);

console.log(response.output_text);

Prompt field reference

Field	Type	Description
idrequired	string	Path of the prompt in the vault, formatted as `project/prompt`.
version	string	Optional tag or commit id. Omit to use the latest published version.
variables	object	Map of variables substituted into `{{block}}` placeholders inside the prompt body.

On Chat Completions and the Messages API

The same prompt object is accepted as an extra-body field (neither upstream API has a native stored-prompt param); messages becomes optional when the referenced prompt provides the conversation. The gateway strips the field before dispatching to the provider.

example.ts

const completion = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "App keeps crashing on launch" }],
  // Caronte extension — or pass it via the SDK's extra-body support
  prompt: { id: "support/triage", version: "production" },
} as never);

tip · Merge semantics

The full per-endpoint behavior — and how the prompt combines with a conversation you already send — is in How the prompt merges into your request below.

How the prompt merges into your request

The behavior is identical across the three endpoints and follows two rules. Send no conversation of your own and the compiled prompt becomes the conversation. Send one and only the prompt's system part is injected — the prompt's user/assistant turns are dropped, because your request already owns the conversation.

Your request	`/v1/responses`	`/v1/chat/completions`	`/anthropic/v1/messages`
Text prompt · no conversation sent	becomes `input` (the user turn)	one `user` message	one `user` message
Text prompt · you send a conversation	prepended to `instructions`	prepended as a `system` message	merged into top-level `system`
Messages prompt · no conversation sent	whole prompt → `input`	whole prompt → `messages`	system → `system`, the rest → `messages`
Messages prompt · you send a conversation	system → `instructions`; other turns dropped	system → `system` message; other turns dropped	system → `system`; other turns dropped

note · Vault-first — and nothing you send is discarded

Wherever the prompt's system content lands next to a system / instructions value you already sent, the vault content goes first and yours is kept below it (<vault>, a blank line, then <yours>). The stored prompt is the base instruction set; your own system text is additive, never overwritten. Same rule on all three endpoints.

note · A text-format prompt has no role of its own

On its own it becomes the user turn, so a stored text prompt is directly runnable. The moment you supply your own conversation it becomes system content instead — your messages are already the conversation, so the prompt can only be an instruction. This inference is the same on every endpoint.

warning · Two Messages-API specifics

Anthropic has no system role inside messages, so a messages-format prompt made of only system/developer turns cannot stand alone on /anthropic/v1/messages — you must then send at least one message (on /v1/responses and /v1/chat/completions those system turns can). And when a prompt carries several system turns, /anthropic/v1/messages concatenates them all while /v1/responses and /v1/chat/completions use the first.

A common pattern

Keep the system message and few-shot examples in the Prompt Vault and pass only the per-call user turn (input on Responses, a user message on Chat Completions and the Messages API). Prompt changes then ship without a code deploy.

tip · Combining with your own system text

Because the vault system is merged vault-first and your instructions / system are preserved, you can store the stable policy in the vault and still add a per-call system note in code — both reach the model, stored prompt first.