OpenAI-compatible API: what it means and what breaks

What the phrase actually promises

The Responses API split is the one that catches people in 2026. Two OpenAI-shaped protocols now exist: Chat Completions, the one everything supports, and Responses, the newer one. The Codex CLI removed the wire_api = "chat" value in version 0.122, so a modern Codex is a Responses client and nothing else. A gateway advertising "OpenAI-compatible" that serves only chat completions will not run Codex, and the error will be an unhelpful protocol or parse failure rather than an honest 404.

The base URL rule that causes most failures

Clients build their own request path from the base you give them. The two families append different things, so the correct base is different, and someone who has just configured one gets the other wrong.

Claude Code

# Bare origin. Claude Code appends /v1/messages itself.
export ANTHROPIC_BASE_URL="https://continuumcode.ai"
export ANTHROPIC_AUTH_TOKEN="cont_sk_..."
export ANTHROPIC_MODEL="claude-opus-5"
# Background turns use a separate small model whose built-in
# default id most gateways do not serve. Pin it.
export ANTHROPIC_SMALL_FAST_MODEL="claude-sonnet-5"

claude

Run /status inside the session to see the base URL the CLI is actually using. If it still shows Anthropic's own endpoint, the shell was open before you exported the variables.

Codex CLI

model = "gpt-5.6-sol"
model_provider = "continuum"

[model_providers.continuum]
name = "Continuum"
base_url = "https://continuumcode.ai/v1"
env_key = "CONTINUUM_API_KEY"
wire_api = "responses"

env_key names a variable, it does not hold the secret, so the value has to be exported in the shell that runs codex. And wire_api has to be "responses": it accounts for more failed Codex-plus-gateway setups than the key, the URL, and the model name combined, because the other three produce errors that name the thing that is wrong.

Cursor

Cursor's Models settings accept a custom OpenAI API key and an overridden base URL. It validates the pair against GET /v1/models when you save, which is why serving that endpoint is not optional. Cursor only pre-lists OpenAI's own model names, so every other model has to be added by hand under Models, Add model. Cursor Tab and Cursor's own Composer models are unaffected and stay on your Cursor plan.

The four things that break

1. Tool calling shapes

This is the big one. Tool calls are the most intricate part of the schema and the least consistently implemented. Symptoms include tool arguments arriving as a string where the client expects an object, parallel tool calls collapsed into one, missing tool_call_id on results, and a client rejecting a replayed message because it carries a field that only belongs on a streamed delta.

2. Streaming

Differences show up in whether the final chunk carries a usage block, whether [DONE] is sent, how errors mid-stream are represented, and whether the first chunk contains the role. Most clients tolerate variation; agent frameworks that reconstruct exact message objects do not. Test with a streamed tool call, not a streamed sentence.

3. Structured output and response format

response_format with a JSON schema is supported unevenly and sometimes silently ignored, which is the worst failure mode: you get valid JSON most of the time and free-form prose on the request that matters. If your pipeline depends on schema enforcement, verify it against the specific gateway rather than the model.

4. Newer and vendor-specific parameters

Reasoning effort, thinking budgets, cache control, service tiers, and per-vendor extensions are the moving edge. Some gateways pass unknown fields through, some drop them silently, and some 400. Silent dropping is the dangerous one, because your reasoning-effort setting simply stops applying and nothing tells you.

# 1. Does the models list answer? Most clients validate against this.
curl -s https://your-gateway.example.com/v1/models \
  -H "Authorization: Bearer $KEY" | head -c 400

# 2. Does a plain completion work?
curl -s https://your-gateway.example.com/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H "content-type: application/json" \
  -d '{"model":"MODEL_ID","messages":[{"role":"user","content":"say ok"}]}'

# 3. Does a TOOL CALL work? This is the one that actually fails.
curl -s https://your-gateway.example.com/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H "content-type: application/json" \
  -d '{"model":"MODEL_ID","messages":[{"role":"user","content":"weather in Paris?"}],
       "tools":[{"type":"function","function":{"name":"get_weather",
       "parameters":{"type":"object","properties":{"city":{"type":"string"}}}}}]}'

Anthropic-compatible, and why some gateways serve both

The Anthropic surface is a smaller contract: POST /v1/messages, a different content-block model, system as a top-level field rather than a message, and anthropic-version as a header. It matters because Claude Code, the Anthropic SDKs, and a growing number of agent tools speak it natively and cannot be pointed at an OpenAI-shaped endpoint without a translator.

If your gateway serves only one surface, LiteLLM is the usual translator in either direction, and the trade is one more hop in the path of every request.

Questions people ask

Keep reading

Continue through LLM gateways and routing with LLM gateway pricing compared: what each one really charges.