AI Gateway features
Code Interpreter
AI Gateway ships a sandboxed Python / JavaScript runtime that the model can drive as a hosted server tool on /v1/responses. The proxy intercepts the tool call, runs it in worker-code-exec, and feeds the result back to the model — your client never sees the inner round-trips. Generated files are uploaded to S3 and surfaced as container_file_citation annotations.
How it works
A quick tour before the examples.
- Turn it on by enabling the code execution tool on your request, just like any other tool.
- The model decides when to use it — for math, data transformations, generating a chart or file, and similar tasks.
- Code runs in a safe sandbox, isolated from your app and your data.
- If the code needs information only your app has, it pauses and asks for it through a normal tool call, then continues once you provide the answer.
- You get the final result — output, errors, and any generated files come back to the model, which uses them to write the answer.
- You can ask the model to read or produce files, and then download them to use in your app.
Minimal example
No containers, no files — let the model run a quick calculation in the sandbox.
example.ts
import { client } from "./client";
const response = await client.responses.create({
model: "gpt-4o-mini",
input: "Compute the mean and standard deviation of [3, 7, 7, 19, 24].",
tools: [
{ type: "caronte:code_execution" } as never,
],
});
console.log(response.output_text);Configuration
The marker accepts a small set of optional fields. They are forwarded verbatim from your request — the model has no input here, so it cannot invent a stale container id or sneak in a tool it was not granted.
| Field | Type | Description |
|---|---|---|
| typerequired | "caronte:code_execution" | Identifies the hosted Code Interpreter tool. |
| container | string | Public container id (cnt_…) the run attaches to. Files already in the container are mounted read-only under ./external-files/, and new files produced by the run are appended to it. |
| file_ids | string[] | List of uploaded file ids (file_…) to mount in the sandbox. If no container is set, an implicit container is created automatically and returned in the result. |
| allowed_tools | string[] | Whitelist of your client-declared function tools to expose as Python/JS stubs inside the sandbox. Default-deny: omit it or pass an empty array and the executed code cannot call any function tool. Only the listed tools become stubs; the model may still call any declared function tool directly. |
tools[] entry
{
"type": "caronte:code_execution",
"container": "cnt_abc123",
"file_ids": ["file_xyz789"],
"allowed_tools": ["advanced_query"]
}Containers
A container is a virtual workspace that holds files made available to a run. TTL is idle-based (default 60 minutes since last_active_at, min 1 min, max 4 h), resets on every run that uses it.
The cnt_* object
| Field | Type | Description |
|---|---|---|
| id | string | Public id (cnt_<hex>) — the one you pass to container. |
| status | "active" | "expired" | "deleted" | Lifecycle state. |
| created_at / last_active_at / expires_at | number | Unix seconds. |
| expires_after | { anchor, minutes } | Idle TTL. anchor is always last_active_at. |
| implicit | boolean | true when the container was auto-created because a run was started with file_ids only. |
REST surface
| Field | Type | Description |
|---|---|---|
| POST /v1/containers | create | Body: { name?, expires_after?: { anchor: "last_active_at", minutes } }. |
| GET /v1/containers | list | Pagination via limit / after. |
| GET /v1/containers/:id | retrieve | Fetch by public id. |
| DELETE /v1/containers/:id | delete | Soft-delete; the run history is preserved. |
| GET /v1/containers/:id/files | list files | Returns both user uploads and files produced by previous runs. |
| POST /v1/containers/:id/files | attach file | Multipart file upload OR JSON { file_id } to attach an already-uploaded file owned by the same API key. |
Example: create a container, upload a file, run on it
example.ts
import { client } from "./client";
// 1. Create a workspace.
const container = await (client as any).containers.create({
name: "monthly-report",
expires_after: { anchor: "last_active_at", minutes: 60 },
});
// 2. Upload a CSV the model will be able to read from ./external-files/.
const file = await client.files.create({
file: await openAsFile("./sales.csv"),
purpose: "user_data",
} as never);
await fetch(`${process.env.CARONTE_BASE_URL}/containers/${container.id}/files`, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${process.env.CARONTE_API_KEY}`,
},
body: JSON.stringify({ file_id: file.id }),
});
// 3. Ask the model to analyse it.
const response = await client.responses.create({
model: "gpt-4o-mini",
input: "Plot total revenue by month from sales.csv and summarise the trend.",
tools: [
{ type: "caronte:code_execution", container: container.id } as never,
],
});
console.log(response.output_text);tip · Implicit containers
If you pass
file_ids without a container, the proxy auto-creates an implicit container and returns its id in the tool result so you can reuse it on later turns.Tool result shape
The JSON object the proxy hands back to the model after a run. You won't parse it yourself unless you read the trace payload, but knowing the shape helps debug what the model saw.
code_execution result
{
"run_id": "cexrun_a1b2c3...",
"container_id": "cnt_abc123",
"stdout": "Mean: 12.0\nStdev: 8.34\n",
"stderr": "",
"exit_code": 0,
"status": "completed",
"duration_ms": 432,
"inner_tool_calls": 0,
"files": [
{
"file_id": "cfile_9e8f...",
"filename": "plot.png",
"mime_type": "image/png",
"size_bytes": 18342,
"container_id": "cnt_abc123"
}
]
}| Field | Type | Description |
|---|---|---|
| run_id | string | Coordinator-issued id (cexrun_…). Also stored in the trace. |
| container_id | string? | The container the run was attached to (real one or implicit one). Absent if the run did not use any container. |
| stdout / stderr | string | Captured from the sandboxed process. |
| exit_code | number | null | Process exit code on completed, null for failed / cancelled. |
| status | "completed" | "failed" | "cancelled" | Run lifecycle outcome. |
| error | string? | Human-readable failure reason (only on failed). |
| duration_ms | number | Wall-clock run time. |
| inner_tool_calls | number | How many client tools the snippet called inside the run. |
| files | GeneratedFile[] | Each entry has file_id, filename, mime_type, size_bytes, and container_id. Download via the management API — the storage URL is internal and never surfaced to clients. |
File annotations on the final response
Files produced by the run surface on the final assistant message as container_file_citation annotations — the same shape OpenAI uses for hosted Code Interpreter. The file body is downloaded from the proxy with the same API key.
response.output[*].content[*].annotations[*]
{
"type": "container_file_citation",
"container_id": "cnt_abc123",
"file_id": "cfile_9e8f...",
"filename": "plot.png",
"start_index": 122,
"end_index": 134
}Direct /v1/code-execution API
For programmatic use cases where you don't want a model in the loop, the same runtime is exposed under /v1/code-execution/runs. The request opens an SSE stream with a small, well-defined event set; you resolve inner tool calls via a POST.
| Field | Type | Description |
|---|---|---|
| run.created | event | Sent immediately. Payload: { runId }. |
| tool_call.required | event | Emitted when the snippet calls a declared client tool. Payload: { runId, calls: [{ call_id, name, arguments }] }. Resolve via POST /v1/code-execution/runs/:runId/tool-results. |
| run.completed | event | Final event on success. Payload: { runId, stdout, stderr, exit_code, duration_ms, inner_tool_calls }. |
| run.failed | event | Final event on failure / cancellation. Payload: { runId, error, stdout, stderr, duration_ms }. |
| [DONE] | sentinel | Last SSE chunk (data: [DONE]), mirrors OpenAI's streaming convention. |
Start a run
bash
curl -N https://<your-caronte>/v1/code-execution/runs \
-H "Authorization: Bearer $CARONTE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"language": "python",
"code": "print(sum(range(10)))",
"tools": []
}'Resolve an inner tool call
bash
curl https://<your-caronte>/v1/code-execution/runs/$RUN_ID/tool-results \
-H "Authorization: Bearer $CARONTE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"results": [
{ "call_id": "<call_id from tool_call.required>", "output": "{\"ok\":true}" }
]
}'Cancel a run
bash
curl -X DELETE https://<your-caronte>/v1/code-execution/runs/$RUN_ID \
-H "Authorization: Bearer $CARONTE_API_KEY"Inner tool calls inside the sandbox
When you declare your own function tools alongside caronte:code_execution, the proxy auto-generates Python/JS stubs the snippet can call. Calling a stub pauses the run and surfaces a regular function_call to the model (or to the direct API caller); the result you provide is handed back as the stub's return value.
snippet the model writes
# advanced_query was declared as a function tool on the request.
# The proxy auto-generates this stub inside the sandbox:
rows = advanced_query(sql="select count(*) from generations")
print(rows)note · Allowed tools
List which client tools the snippet can call in
allowed_tools on the marker. This is default-deny: if you omit allowed_tools or pass an empty array, the snippet cannot call any function tool. The model can still invoke any declared tool directly through standard function calling — the whitelist only governs what runs inside the sandbox.Pre-installed libraries
The sandbox image ships with a curated data-science / document-generation toolkit so the model can produce files without going through pip install (which is disabled — outbound HTTP is blocked anyway). Import any of these directly from your snippet.
| Field | Type | Description |
|---|---|---|
| Data & scientific computing | python | pandas, numpy, scipy, scikit-learn, sympy, networkx. |
| Plotting & images | python | matplotlib, seaborn, Pillow (imported as PIL). |
| Spreadsheets | python | openpyxl, xlsxwriter — pair with pandas.to_excel(...) for quick exports. |
| python | reportlab, pypdf, fpdf2 (imported as fpdf) for generation and manipulation. | |
| Office documents | python | python-pptx (imported as pptx), python-docx (imported as docx). |
| Parsing & formatting | python | lxml, beautifulsoup4 (imported as bs4), pyyaml (imported as yaml), tabulate. |
| Synthetic data | python | Faker (imported as faker) — locale-aware fake names, addresses, companies and dates for fixtures and demo datasets. Seed with Faker.seed(n) for reproducible output. |
| HTTP client | python | requests is importable but the sandbox blocks outbound network — calls will fail at connect time. Use it only for offline payload shaping. |
| Standard library | python | Full CPython 3 stdlib — csv, json, io, pathlib, datetime, statistics, re, etc. No extra install needed. |
| JavaScript runtime | javascript | language: "javascript" runs on Bun — only the Bun + Web standard APIs are available (fetch, Bun.file, fs, …). No third-party npm packages are pre-installed and bun add is disabled. |
Limits & defaults
Practical numbers you might hit.
| Field | Type | Description |
|---|---|---|
| timeout_ms | number | Wall-clock cap per run. Default 30 000, max 300 000 (5 minutes). Passed as a parameter on the code_execution tool call. |
| container TTL | minutes | Default 60, min 1, max 240 (4 hours). Resets on every run that touches the container. |
| external files | read-only | Files are mounted under ./external-files/ inside the sandbox. The snippet can read them but cannot mutate them in place. |
| upload size | 50 MiB | Hard cap per file on POST /v1/files and the multipart variant of POST /v1/containers/:id/files. Uploads above this return 400 before the body is buffered, so oversized inputs never reach the sandbox. |
| external files per run | 50 files | Worst-case 50 files are materialised under ./external-files/ per run; any attached beyond that are skipped with a worker log line. Each individual file is additionally capped at 50 MiB (defense-in-depth against tampered metadata). |
| generated files per run | 50 files / 50 MiB each | The worker scans the top level of the sandbox after the run and uploads at most 50 output files, each capped at 50 MiB. Anything larger or beyond the count is dropped with a log line and never surfaced as a container_file_citation. |