OpenRouter has changed how it chooses an inference provider by default. Its documentation now says that for each request it compares "what a good response would actually cost you at each one, not by list price alone," using measured price, throughput, time to first token, rate limits, reliability and tool-call quality. The company is reported to have said early tests on open-weight models showed about 20 percent higher throughput and time to first token, roughly 40 percent fewer fallbacks and about 60 percent fewer rate-limit errors. The docs confirm the design; we could not locate the original posts for the gains, which would be OpenRouter's own claims.
If you call open-weight models through OpenRouter, this changes who serves your tokens even if you change no code. This post explains the logic, how to opt out, and how to check the numbers on your own traffic. For earlier OpenRouter coverage, see our write-ups of Armor, its tool-call safety classifier and a model that launched half price on the platform.
Claimed vs. verified
| Claim | Status | Source |
|---|---|---|
| Default routing weighs more than list price | Confirmed in the docs | OpenRouter provider routing docs |
| Inputs: effective price, throughput, TTFT, 429s, reliability, quality | Confirmed in the docs | Same page |
| About 20% faster throughput and TTFT on open-weight models | Reported company claim, early testing; we could not locate the original posts | Unverified |
| About 40% fewer fallbacks | Reported company claim | Unverified |
| About 60% fewer rate-limit errors | Reported company claim | Unverified |
| Providers that deliver more value win more traffic | Consistent with the docs' weighted shares | Docs |
How the default routing works, per the docs
The documentation lists six measured inputs, all taken from recent production traffic so routing reacts within minutes:
- Effective price: what the provider actually charges once its real cache-hit rate and typical input/output token mix are included.
- Throughput: end-to-end output tokens per second. A faster provider can charge more and still win.
- Time to first token: how long you wait before streaming starts.
- Rate limits: how often the provider rejects requests with a 429 because it is out of capacity.
- Reliability: how often requests fail because of the provider.
- Quality: whether the provider's tool calls come back well formed.
Providers close to the best share most of the traffic and each request picks one at random, weighted by those shares. Providers far behind still get a small share so routing can notice when they improve. On top of that, the docs say a provider whose errors or 429s are far worse than the model's other providers gets no traffic until it recovers; routing learns how much load each provider can take before returning 429s and sends the rest elsewhere; and a burst of 429s or a latency jump briefly lowers a provider's share. If a request fails, the retry is picked the same way from the remaining providers, and follow-ups in a conversation stay with the same provider while its prompt cache is warm. OpenRouter's prompt caching docs describe this as sticky routing, which does not apply when you set a manual provider order. A 429 is the standard "rate limited" status in OpenRouter's error reference, which advises honoring any Retry-After header.
The design means providers that deliver more value win more traffic, so every provider has a reason to keep improving.
Why open-weight models are the target
The same open-weight model, say a Qwen, a GLM or a DeepSeek release, is often served by a dozen hosts with different quantization, hardware, caching and capacity. Quality and speed vary widely even though the model name is identical. Sorting by price alone sends traffic to the cheapest host, which may be quantized more aggressively, throttled, or poor at structured output. For agents that depend on tool calls, a malformed call costs a retry and often a failed run.
That is why the quality input matters. It turns a hidden cost of cheap hosting into a visible score. It also gives hosts a reason to invest in reliability and caching, not only to undercut. For background on how developers choose between hosts, see our coverage of Vercel AI Gateway's open-model volume and startups saving money with open-weight models.
What this replaces
OpenRouter's earlier default, as described in third-party guides and its own older documentation, was price-weighted: spread requests across the cheaper providers for a model, deprioritize any that had a significant outage in the last half minute, and fall back to another host on failure. Developers who wanted something else had to opt in with a sort order. The new design keeps the weighted-random spread and the fallback behavior but changes the score used to weight providers, and it adds explicit penalties for 429s and malformed tool calls.
Two consequences are worth noting. First, price is still an input, but it is now the effective price that includes cache hits and token mix, which can reorder providers whose list prices look similar. A host with strong prompt caching can look more expensive on paper and cheaper in practice for agent workloads that resend large prefixes. Second, because the score is measured from live traffic, a provider's share can move within minutes. That is good for avoiding a degraded host, but it also means the same request can land on different providers from one hour to the next, so results can vary slightly in latency and sometimes in output if hosts quantize differently.
For teams with evals, this makes pinning a quantization level more important than it used to be. For teams without them, it is a prompt to build a small regression set. The cost of a bad host used to show up as a bill; now it is more likely to show up as a shift in quality that nobody notices until a user complains.
How to keep control
The default is a convenience, not a requirement. Per the docs, default routing is bypassed when you do any of the following:
- Set
sortto price, throughput or latency, or set a default provider sort in your account settings. - Append
:floor(cheapest, with flex-tier endpoints eligible) or:nitro(fastest, with priority-tier endpoints eligible) to the model slug. - Set
orderto a list of provider slugs to try in sequence. - Use your own provider key.
If you set only only or ignore and two or more providers remain, default routing picks among them. Other controls remain available: quantizations to filter by precision, zdr to restrict to zero-data-retention endpoints, data_collection to deny providers that may store data, max_price to cap what a request can cost, and preferred_min_throughput or preferred_max_latency to deprioritize slow endpoints without excluding them.
A reasonable setup for most teams: leave the default on for general workloads, pin quantizations if your evals are sensitive to precision, and use an explicit order or only list for anything with compliance requirements.
How to check the 20/40/60 claims yourself
OpenRouter's numbers come from its own early testing on open-weight models, and your mix of models, regions and prompts will differ. To test:
- Log provider, model, time to first token, total latency, status code and tokens for a week before and after, if you have the history.
- Count fallbacks (requests where the final provider differs from the first attempt) and 429s separately from other errors.
- Compare cost per successful task, not cost per token. A routing change that raises token price can still lower cost per task if retries fall.
- For agents, track the share of tool calls that fail to parse and the number of retries per task.
- Run a fixed prompt set against two or three providers using
onlyto see how the router's choices line up with your own measurements.
This matches the approach we recommend in our notes on cost per task versus price per token and in the LangChain router case study.
What we could not verify
We found the design in OpenRouter's docs but no technical write-up that publishes the test methodology, sample size or the baseline for the 20, 40 and 60 percent figures. "About 20 percent" could mean a median or a mean, across all providers or across a sampled set of models. Treat the numbers as directional until OpenRouter or an independent party publishes the data. A related risk is concentration: if the weighted shares favor the same few providers, smaller hosts may lose traffic and the market could narrow over time, which is a question for the next few months of data.
What this means for builders
If you build agents on open-weight models, routing quality is now part of your reliability story. Expect fewer surprise 429s and fewer provider flips mid-task, but keep fallbacks and your own evals. If you are a host, tool-call correctness is now a measured input to traffic share, so test your function-calling output carefully. If you resell or monitor usage, note that the provider behind a model name can change with load.
Related reading
Details are accurate as of October 29, 2026; routing logic and numbers can change.