The server had an error while processing your request: OpenAI 500, 502, 503 and 504

The four codes, and which two OpenAI actually documents

Start with what is in the documentation, because the absences are as informative as the entries.

The two undocumented codes are the interesting ones. OpenAI serves the API through its own infrastructure and reports internal failures as 500 or 503. A 502 or 504 reaching your client, particularly with an HTML body rather than JSON, almost always means something in between generated it: a corporate proxy, an API gateway, a service mesh, a CDN, or a load balancer with an idle timeout shorter than your request. The tell is the body. A JSON error object with a type field came from OpenAI. An HTML page did not.

Slow Down is not overload

Both 503s carry the same status code and mean opposite things, and only one of them is safe to retry. The distinction is worth the paragraph because getting it backwards actively makes your situation worse.

That is not generic backoff advice, it is a specific protocol with a specific duration. The docs add that this can occur on Pay-As-You-Go models, which are shared across all OpenAI users, and point at the Scale Tier for guaranteed capacity.

Retrying properly

The SDKs already do most of this. Both the Python and Node clients state it identically: certain errors are automatically retried two times by default with a short exponential backoff, covering connection errors, 408 Request Timeout, 409 Conflict, 429 Rate Limit, and anything 500 or above. Retry-After is honoured. Timed-out requests are retried too.

from openai import OpenAI

# Defaults, for reference: max_retries=2, timeout=600s total / 5s connect,
# initial retry delay 0.5s, max retry delay 8s, Retry-After capped at 120s.

interactive = OpenAI(max_retries=1, timeout=45.0)   # a human is waiting
batch       = OpenAI(max_retries=8, timeout=900.0)  # nobody is waiting

When you write the loop yourself, OpenAI's own guidance is explicit about jitter: honour Retry-After when present, and when it is absent, "use exponential backoff with jitter and limit the number of retries".

• Retry 500, 502, 503 overload, and 504. Bounded, jittered, capped.
• Do not retry 503 Slow Down at the same rate. Cut the rate first, then resume.
• Never retry a 4xx other than 408, 409, and 429. They cannot succeed and the retries just delay the real error.
• Never retry the billing 429s. OpenAI states directly that retrying billing, spend, or quota errors will not restore access.
• Cap total elapsed time, not just attempt count. Eight attempts with a long tail can exceed any user's patience and any upstream timeout.

The agent problem: retrying a call that already did something

This is the part that separates an agent workload from a chat completion, and it needs saying plainly.

Why it matters: a 500 tells you the request failed, not that nothing happened. In a chat completion the worst case is a duplicated generation, which costs tokens and nothing else. In an agent loop, the model may have already emitted a tool call that your executor already ran. Retrying the API call replays the turn, the model calls the tool again, and your side effect happens twice. Two payments, two branches, two emails.

import hashlib, json

def tool_key(turn_id: str, name: str, args: dict) -> str:
    payload = json.dumps(args, sort_keys=True)
    return hashlib.sha256(f"{turn_id}:{name}:{payload}".encode()).hexdigest()

def execute(turn_id, name, args):
    key = tool_key(turn_id, name, args)
    cached = results.get(key)
    if cached is not None:
        return cached           # the retry sees the first result, not a second execution
    out = TOOLS[name](**args)
    results.put(key, out, ttl=86400)
    return out

• Key on the turn, not the request. A retried API call is a new HTTP request but the same logical turn.
• Classify your tools. A file read is safely repeatable. A payment, a merge, an email, and a delete are not. Only the second group needs the cache, and only that group is worth the complexity.
• Persist the ledger. An in-memory dict does not survive the process restart that a 500 storm often triggers.
• Use background mode for long work. OpenAI documents background: true with polling for exactly this: the work continues server-side and you resume against a response ID rather than replaying a whole turn. Cancellation is idempotent even when nothing else is.

Is it OpenAI, or is it you?

Ninety seconds of triage, in order, before you change anything.

curl -sS -o /tmp/body -w '%{http_code} %{content_type}\n' \
  https://api.openai.com/v1/responses \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.6-luna","input":"ping","max_output_tokens":16}'

head -c 300 /tmp/body

Questions people ask

Keep reading

Continue through Errors and fixes with RESOURCE_EXHAUSTED on the Gemini API: which of the two 429s you have.