An LLM router picks the model for each request. Cost routing sends easy work to cheap models, capability routing matches task to model strength, latency routing picks the fastest provider serving a model, and fallback routing handles failure. The decision is made either by static rules you write (fast, predictable, stale) or by a classifier that grades the prompt (adaptive, opaque, adds latency). OpenRouter, claude-code-router, and Continuum Auto Model Mode are three real implementations that make three different tradeoffs.
- Four kinds of routing: cost, capability, latency, fallback. They are not the same feature.
- Static rules are auditable and go stale. Classifiers adapt and cost you a call before the call.
- The biggest real saving is not model choice, it is not sending the request twice.
- A router that hides which model answered is a router you cannot debug. Demand a stamp.
- Routing on context length is the one rule that pays for itself immediately.
- Routing to the cheapest model that clears a quality floor beats routing to the cheapest model.
The four kinds of routing
| Kind | The decision | Typical trigger |
|---|---|---|
| Cost routing | Which model, out of several that could answer, is cheapest for this request | Prompt length, task classification, a per-team budget |
| Capability routing | Which model is good enough at this kind of work | Reasoning-heavy versus mechanical, code versus prose, tool use |
| Latency routing | Which provider of the same model answers fastest right now | Live throughput and time-to-first-token measurements |
| Fallback routing | What answers when the first choice returns an error | 429, 5xx, 529, timeout, content policy refusal |
Latency routing is the one people forget exists, and it is invisible unless a model is served by more than one provider. OpenRouter is explicit about this: by default it excludes providers with significant outages in the last 30 seconds and then weights by inverse square of price, and you can override that with sort: "throughput", sort: "latency", or sort: "price". The shortcuts :nitro and :floor appended to a model slug are the same two decisions in one token.
How the decision gets made: rules versus grading
Static rules
A table you write. if tokens > 60,000 use the long-context model. if the task is "summarise a diff" use the cheap model. Fast, free, auditable, and completely predictable, which is why almost every production system starts here and many stay.
claude-code-router is the clearest example of the pattern applied to a coding agent. Its configuration names a model per request class rather than per request: default, background, think, and longContext, with longContextThreshold defaulting to 60,000 tokens. The router does not evaluate your prompt; it looks at which slot the agent asked for and which threshold the context crossed.
{
"Router": {
"default": "provider,model-for-general-coding",
"background": "provider,cheap-fast-model",
"think": "provider,reasoning-model",
"longContext": "provider,long-context-model",
"longContextThreshold": 60000
}
}The failure mode of static rules is staleness. The rule that said "use the mid-tier model for classification" was correct when the mid-tier model cost $3 per million input tokens; it is a bad rule now that Claude Sonnet 5 is $2 in and $10 out as its permanent standard rate. Nobody revisits routing tables, which is the real argument for the other approach. If you do keep a table, re-derive it from a current quality signal rather than from memory: our coding-model leaderboard tracks measured pass rate and cost per task across the models a router picks between.
Model-graded and classifier routing
Instead of a rule, something reads the prompt and decides. OpenRouter's openrouter/auto classifies the prompt into roughly 30 task types such as code:debugging or math, ranks models by real-world spend share within that category over a seven-day window, applies your cost-tier preference, and degrades to a default model set if the classifier is unavailable.
{
"model": "openrouter/auto",
"messages": [{"role": "user", "content": "Explain quantum entanglement"}],
"plugins": [{"id": "auto-router", "cost_tier": "xhigh"}]
}A worked example: routing across roles, not requests
Continuum's Auto Model Mode is worth walking through because it routes on a different axis from the other two: not "which model for this request" but "which model for this role in this task".
Triage the prompt before routing anything
The prompt gets a score. Length pushes it up past 400 and again past 1,200 characters, as do architecture words (refactor, migrate, protocol, end-to-end), three or more file paths, and a numbered structure. Scoped-fix words like "typo" pull it back down. Below the threshold, one fast model takes the whole turn, because a three-model pipeline on "fix this typo" costs four times as much for a worse answer.
Assign three roles above the threshold
Planner, executor, verifier. Each model family carries five scores (cost, intelligence, taste, speed, agentic ability) and each role reads them differently: the planner takes highest intelligence then taste, the executor weights intelligence three times against agentic ability because a model that reasons well but cannot drive tools is useless, and the verifier takes intelligence first and prefers a different provider from the executor.
Prefer the rail that costs less
When one model family is reachable on more than one rail, a first-party subscription you already pay for beats a metered API key, which beats an aggregator. This is the routing decision with the largest financial effect and almost nobody implements it, because it requires knowing what the user is already paying for.
Stamp what actually ran
Every transcript step records the role, the model id, the provider, the rail it billed, the effort level, and the cost. When a fallback fires, the stage is re-stamped with the model that really ran and what it fell back from. A routing choice you disagree with can be traced to the exact scoring snapshot that made it.
Does routing actually save money?
Sometimes, and less than the pitch decks claim. Three honest observations.
- The spread is real. At August 2026 list prices, Claude Haiku 4.5 is $1 in and $5 out per million tokens while Claude Opus 5 is $5 and $25, and GPT-5.6 Luna is $0.20 and $1.20 against GPT-5.6 Sol at $5 and $30. Routing the genuinely easy 60% of a workload down one tier is a 40% to 70% saving on that slice.
- Downgrading agentic work is a false economy. A cheaper model that needs three attempts costs more than an expensive model that needs one, and it costs your afternoon as well. This is why routing to "the cheapest model that clears an intelligence floor" is a different rule from routing to "the cheapest model", and only the first one works.
- Caching usually beats routing. Prompt caching on Anthropic reads at 0.1x the base input rate. If your system prompt is 20,000 tokens and you send it forty times an hour, caching saves more than any model swap will, and it does not change which model answers.
The other reason to route has nothing to do with money: capacity. Spreading a workload across two providers means a rate limit on one is a slowdown rather than an outage, which is failover thinking applied before the failure.
Questions people ask
What is an LLM router?
A component that chooses which model answers each request. It can route on cost (cheap model for easy work), capability (match task to model strength), latency (fastest provider currently serving that model), or failure (what runs when the first choice errors). Most gateways contain a router; a router on its own is a much smaller thing than a gateway.
How does model routing decide which model to use?
Either by static rules you write, such as "over 60,000 tokens use the long-context model", or by a classifier that reads the prompt and scores it. Rules are fast, free, and auditable but go stale as prices change. Classifiers adapt but add a decision before the answer and are harder to debug.
What is openrouter/auto?
OpenRouter's automatic model selection. It classifies your prompt into roughly 30 task types, ranks models by real-world spend share for that category over a seven-day window, applies your cost tier preference, and falls back gracefully to a default model set if the routing infrastructure is unavailable.
Does an LLM router save money?
On mixed workloads with a lot of easy requests, yes: the price spread between tiers is roughly five to ten times. On agentic coding work it often does not, because a cheaper model that needs three attempts costs more than a stronger one that needs a single pass. Prompt caching usually saves more than routing does.
Can I route Claude Code to a different model?
Yes, through ANTHROPIC_BASE_URL pointed at a gateway, or through a purpose-built tool such as claude-code-router which routes per request class (default, background, think, longContext). The tradeoffs, including the unofficial-tool risk, are covered in the claude-code-router guide.
Sources
Every figure above was read from these pages on August 2026. Vendors reprice without notice; if you find a stale number, tell us.