AI Gateway features

Monitoring & traces

Every request that hits AI Gateway is logged as a generation under a trace, which can be grouped into a session. You don't need to do anything to enable this — but if you set a few optional fields in metadata, the dashboard becomes a lot more useful (related turns grouped together, human-readable names in the traces list, end-to-end conversation views, etc.).

The trace model

AI Gateway uses a three-level hierarchy: session → trace → generation. Each level has an id (for grouping) and an optional name (for display).

Generation — a single API call (one chat completion, one response, one embedding, …). This is what you see as a row in the traces list.
Trace — a group of generations that belong to the same logical request (e.g. the multiple turns of one conversation, or a tool-use loop).
Session — a group of traces that belong together at a higher level (e.g. one user's back-and-forth across several conversations).

Metadata fields

Set any of these as string properties of metadata on the request body — they are optional and validated by the proxy. Anything else you put in metadata is forwarded verbatim and shown on the trace.

Field	Type	Description
trace_id	string	Groups multiple generations under the same trace. If absent, AI Gateway generates a UUID per generation (one trace = one generation). Also used as the PII masking session key, so reusing it keeps the same redacted ↔ original bindings across turns.
trace_name	string	Display name for the trace. Defaults to the API key's name.
generation_name	string	Display name for this generation. Defaults to a per-endpoint label (`Chat Completion`, `Responses`, `Embedding`, …).
session_id	string	Groups multiple traces under the same session. Defaults to the resolved `trace_id`.
session_name	string	Display name for the session. Defaults to `trace_name`.

note · They are nested under metadata

These fields are not top-level request parameters — they live inside the OpenAI standard metadata object, which is the same place you put your own arbitrary key/value pairs.

tip · Or send them as headers

All five fields can alternatively be sent as the X-Trace-Id, X-Trace-Name, X-Generation-Name, X-Session-Id and X-Session-Name request headers — handy on the multipart audio/image endpoints where nesting a metadata object in the form is awkward. The header wins over the same field in the body. See Custom headers.

Single-turn (no grouping)

The minimum useful labels: give your trace a human name. Every call without trace_id gets a fresh trace.

example.ts

import { client } from "./client";

await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello" }],
  metadata: {
    trace_name: "support-bot",
    generation_name: "greeting",
  },
});

Multi-turn conversation (one trace)

Generate a trace_id at the start of the conversation and reuse it on every turn — all generations land on the same trace and the dashboard renders them as a single timeline.

example.ts

import { randomUUID } from "node:crypto";
import { client } from "./client";

const traceId = randomUUID();
const meta = {
  trace_id: traceId,
  trace_name: "support ticket #4291",
};

await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "My app keeps crashing." }],
  metadata: { ...meta, generation_name: "classify" },
});

// …later, same conversation
await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [/* full history */],
  metadata: { ...meta, generation_name: "draft-reply" },
});

Sessions (cross-conversation grouping)

Use session_id to group several traces that share a higher-level context — typically the same end user across multiple conversations. Each conversation still gets its own trace_id.

example.ts

import { randomUUID } from "node:crypto";
import { client } from "./client";

const sessionId = "user_42";
const sessionName = "Anna Bianchi";

// First conversation
await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "How do I reset my password?" }],
  metadata: {
    session_id: sessionId,
    session_name: sessionName,
    trace_id: randomUUID(),
    trace_name: "password-reset",
  },
});

// Different conversation, same user
await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Cancel my subscription." }],
  metadata: {
    session_id: sessionId,
    session_name: sessionName,
    trace_id: randomUUID(),
    trace_name: "cancel-subscription",
  },
});

Default behavior

When some fields are missing, the proxy fills them in deterministically. Knowing the defaults is enough to predict how a generation will land in the dashboard.

Field	Type	Description
trace_id	fallback	Random UUID generated per request.
trace_name	fallback	API key's name.
generation_name	fallback	Per-endpoint label — `Chat Completion`, `Responses`, `Embedding`, …
session_id	fallback	Resolved `trace_id`.
session_name	fallback	Resolved `trace_name`.

warning

All five fields must be string when present — passing a number, object, or array returns a 400. UUIDs are not required, any non-empty string works.

Attributing to an end user

Beyond the trace fields, a generation can be attributed to the end user it was made for — send it as the body user field or the X-User header. The same identifier drives force_user and per-user budgets/caps.

note · Full reference on its own page

Resolution order, the default-user fallback, force_user enforcement and per-user budgets/caps are documented in End-user identifier.

Disabling payload storage (X-No-Log)

By default AI Gateway stores the request and response bodies of every generation so you can inspect inputs/outputs on the trace. When a request can't leave its payload at rest — sensitive data, compliance, or just noisy internal traffic — set the X-No-Log header and the payload is never persisted.

This is a request header, not a metadata field. The generation is still recorded — model, token counts, cost, latency, budget/cap accounting and the metadata you sent all land in the dashboard as usual. Only the request and response bodies are skipped: nothing is uploaded to S3, and the trace shows payload not stored where inputs/outputs would normally appear.

Field	Type	Description
X-No-Log	header	Truthy values — `1`, `true`, `yes`, or the bare presence of the header — disable payload storage. `0`, `false`, `no` and an empty value are treated as off (the default). Applies to every endpoint.

example.ts

import { client } from "./client";

// Per call — the body is not stored, but cost/usage/latency still are.
await client.chat.completions.create(
  {
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Contains sensitive data…" }],
    metadata: { trace_name: "pii-heavy-job" },
  },
  { headers: { "X-No-Log": "1" } },
);

bash

curl https://<your-caronte>/v1/chat/completions \
  -H "Authorization: Bearer $CARONTE_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-No-Log: 1" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{ "role": "user", "content": "Contains sensitive data…" }]
  }'

warning · The payload is gone for good

Skipped payloads are never written, so they can't be recovered later — the trace will permanently read payload not stored. To apply it to all traffic set it once via defaultHeaders on the client.

Where to view traces

Per-project trace lists live under /projects/{project_id}/traces. Drill into a trace to see all of its generations, their inputs/outputs, latency, cost, and the metadata you sent.

tip

You can also browse traces from a key's detail page — reach it from its project — and from the monitoring dashboards.