Google's own framing on this one is unusually blunt for a model-marketing account: "You don't always need a frontier model." That's Google Gemma's pitch for a new benchmark from AlphaSense, an AI-powered market intelligence platform, comparing answer quality against cost per question across a spread of open-source and proprietary models — and Gemma 4 31B is the headline result.
TL;DR
| Question | Answer |
|---|---|
| What was benchmarked? | Answer quality vs. median cost per question, for RAG-based search |
| Who ran it? | AlphaSense, an AI market intelligence platform |
| Gemma 4 31B's quality score? | ~1.35 — nearly identical to Sonnet 5 |
| Gemma 4 31B's cost? | ~$0.01 per question — roughly 40x cheaper than Sonnet 5 |
| Highest quality overall? | GPT-5.6 Sol (~1.75), then Opus 4.8 (~1.6) — both far more expensive |
| What's the takeaway? | Near-frontier quality at commodity pricing unlocks high-volume use cases frontier models can't touch economically |
Chart: AlphaSense, via Google Gemma's post on X.
What the chart actually shows
AlphaSense's chart plots answer quality (y-axis) against median cost per question in USD, on a log scale (x-axis, cheaper to the right) for its own RAG-based search product, with a Vector-RAG baseline fixed at 1.0 as the floor for comparison. Every model tested scores above that baseline, but where they land on the cost axis is where the story is.
GPT-5.6 Sol leads on raw quality at roughly 1.75, clustered with Opus 4.8 (~1.6) and Kimi K3 (~1.45) in the "expensive but excellent" region — median costs from a few cents up to over a dollar per question. Sonnet 5 sits at roughly 1.35 quality near the $0.10 mark, in line with Haiku 4.5 and GLM-5.2 nearby.
Gemma 4 31B breaks the expected pattern: it lands at almost exactly Sonnet 5's quality score (~1.35), but out at the chart's cheapest edge — roughly $0.01 per question, on the order of 40x cheaper than Sonnet 5's position on the same axis. Nothing else on the chart combines that quality tier with that price point; Inkling, another open-source entry, is similarly cheap but scores meaningfully lower on quality (~1.25).
Why cost-per-question is the right lens here
Most model comparisons rank by raw capability — coding benchmarks, reasoning evals, agentic tool-use scores. AlphaSense's chart is deliberately answering a different, more commercially blunt question: for a specific, high-volume workload, what do you actually pay to hit a given quality bar?
That framing matters because RAG search — the workload this benchmark measures — is exactly the kind of task run at massive volume inside a real product: every user query against a knowledge base is one model call, and at enterprise scale that's thousands to millions of calls a day. A model that's marginally "better" per Google's own admission but costs 40x more per call doesn't just cost more — at high enough volume, it can make the entire feature uneconomical to ship, independent of whether the answers are slightly better.
Google's Gemma account states this directly: Gemma's advantage isn't winning the quality race outright, it's high cost-efficiency and low latency that "unlocks high-volume use cases that are uneconomical with larger models." That's a materially different pitch than "our model is the best" — it's "our model is good enough, at a price that makes the product possible at all."
The caveats worth keeping in mind
This is one benchmark, on one company's RAG pipeline, measuring one task type. A few things worth weighing before treating "Gemma 4 31B ≈ Sonnet 5" as a general claim:
- It's a single workload. RAG-based search answer quality is not the same as coding ability, agentic tool-calling reliability, or long-context reasoning — domains where the quality gap between a 31B open model and a frontier model like Sonnet 5 or GPT-5.6 Sol may look very different. See our top 10 open-weight models for a laptop roundup for how these same models stack up on other tasks.
- "Answer quality" is a constructed metric. AlphaSense's own scoring methodology — however rigorous — is still one company's judgment of quality, not a universally agreed benchmark like MMLU or SWE-bench.
- Cost figures depend on deployment. Median cost per question reflects AlphaSense's specific usage pattern (query length, retrieved-context size, output length); your own cost-per-query will differ based on how you call the model. Our guide to how AI token pricing actually works covers the mechanics behind why cost varies this much between models in the first place.
- This is exactly the argument for model routing, not model replacement. The finding isn't "always use Gemma," it's "route high-volume, quality-tolerant workloads to the cheap model and reserve frontier spend for tasks that need it" — the same logic behind AT&T's reported 56% coding-cost savings from model routing.
Where Gemma 4 31B fits in Google's lineup
Gemma 4 31B isn't a new model as of this benchmark — it's the same model Cerebras clocked at 1,800+ tokens per second for multimodal inference earlier this year, positioning it as both fast and, per this new data, cheap relative to its quality tier. It sits in a broader Gemma 4 family that includes the 12B multimodal local model, the E4B variant powering on-device iOS automation, and July's flash-attention and tool-calling update — all part of Google's consistent pitch that Gemma exists to be the practical, deployable tier beneath Gemini, not a frontier competitor.
Related on explainx.ai
- Gemma 4 31B on Cerebras: 1,800+ TPS, the fastest multimodal inference yet
- How AI token pricing actually works
- AT&T reported 56% coding-cost savings from model routing
- Top 10 open-weight models for a laptop
- Gemma 4 July 2026 update: flash attention, tool calling, vision
- Choosing open-weight vs. closed AI models
Primary source: AlphaSense benchmark, via Google Gemma on X
Reflects the AlphaSense benchmark as shared by Google Gemma on August 21, 2026. Cost and quality figures are AlphaSense's own measurements on its RAG search pipeline and may not generalize to other workloads.