Getting started

Client configuration

Point the official OpenAI SDK at AI Gateway by overriding two settings: baseURL and apiKey. Every example in these docs reuses the client built here.

Synchronous client

client.ts

import OpenAI from "openai";

export const client = new OpenAI({
  apiKey: process.env.CARONTE_API_KEY!,
  baseURL: process.env.CARONTE_BASE_URL ?? "https://<your-caronte>/v1",
});

Async client

Same two settings, async variant — useful for agents, web servers, or anything streaming.

client.ts

import OpenAI from "openai";

// The OpenAI JS SDK is async by default; the same instance works for
// both blocking and streaming calls.
export const client = new OpenAI({
  apiKey: process.env.CARONTE_API_KEY!,
  baseURL: process.env.CARONTE_BASE_URL ?? "https://<your-caronte>/v1",
});

If you'd rather not pass the settings explicitly, the OpenAI SDK also reads OPENAI_API_KEY and OPENAI_BASE_URL from the environment, so you can drive your existing code through AI Gateway by just exporting them.

.env

OPENAI_API_KEY=sk_xxxxxxxxxxxx
OPENAI_BASE_URL=https://<your-caronte>/v1

Custom headers

AI Gateway reads a few of its own request headers on top of the standard OpenAI ones. Set them once on the client via defaultHeaders (applied to every call) or per request via the SDK's request options. The one you'll reach for most is X-No-Log.

client.ts

import OpenAI from "openai";

// Applied to every request from this client.
export const client = new OpenAI({
  apiKey: process.env.CARONTE_API_KEY!,
  baseURL: process.env.CARONTE_BASE_URL ?? "https://<your-caronte>/v1",
  defaultHeaders: { "X-No-Log": "1" },
});

// …or just for a single call (overrides the client default):
await client.chat.completions.create(
  { model: "gpt-4o-mini", messages: [{ role: "user", content: "Hi" }] },
  { headers: { "X-No-Log": "1" } },
);

note · X-No-Log — skip payload storage

When this header is set on any endpoint, AI Gateway still records the generation (cost, tokens, latency, budget/cap accounting and your metadata) but does not store the request and response bodies — the trace shows payload not stored instead of inputs/outputs. See Monitoring & traces for the full behavior.

The full set of request headers AI Gateway reads. All work on every endpoint — including the multipart ones (/v1/audio/transcriptions, /v1/images/edits) where nesting fields in the body is awkward — and each takes precedence over the equivalent body field when both are sent.

Field	Type	Description
X-No-Log	header	Skip persisting the request/response bodies for this request. See Monitoring & traces.
X-Trace-Id	header	Header form of `metadata.trace_id` — groups generations under one trace.
X-Trace-Name	header	Header form of `metadata.trace_name` — display name for the trace.
X-Generation-Name	header	Header form of `metadata.generation_name` — display name for this generation.
X-Session-Id	header	Header form of `metadata.session_id` — groups traces under one session.
X-Session-Name	header	Header form of `metadata.session_name` — display name for the session.
X-User	header	End-user identifier — header form of the body `user` field. Drives `force_user` enforcement and per-user budgets/caps. See End-user identifier.

Request size limits

Each endpoint caps the total request body. The JSON APIs allow larger bodies so inline base64 images fit; the multipart upload APIs cap higher for file payloads.

Endpoint	Max request body	Notes
/v1/chat/completions	32 MB	Inline base64 images count toward this total.
/v1/responses	32 MB	Inline base64 images count toward this total.
/v1/embeddings	32 MB	—
/v1/code-execution/runs	32 MB	Submitted code plus any inline tool results.
/v1/images/generations, /v1/images/edits	50 MB	Multipart uploads (source image + optional mask).
/v1/audio/transcriptions	50 MB	A single audio file must not exceed 25 MB.
/v1/files	50 MB	Per uploaded file.

warning · Over-limit requests return 413

A body above the cap is rejected with 413 and an invalid_request_error (code request_too_large) before the request reaches the provider. These are AI Gateway's ceilings — the upstream model provider may enforce stricter per-item limits (for example, some providers cap a single image at ~5 MB). To send inputs larger than the request cap, upload them through /v1/files instead of inlining them as base64, or split the input.

Synchronous client

Async client

Environment variables

Custom headers

Request size limits