explainx.ai350K+ learn from us
TrendingAI News TodayPathwaysSkills
Pricing
explainx.ai

Upskill in AI — 16 free pathways, live workshops & bootcamps, and 50+ courses from practitioners. Plus the skills, tools, and MCP servers to practice on.

follow us

follow on google

Add explainx.ai as a preferred source

corporate training

support@explainx.ai

get started

Find your pathTake Free Evaluation

community

Join the community

learn

mind: share how you thinkpathways — start freeworkshopsbootcampscoursescompare Explainxcertificationsmock testsexplainx universitycorporate traininglearn skills & mcp

discover

skillsmcp serversexplainx mcpestacktoolsmdx readeragentsllmsdesignsdictionarypeopleagi trackerfelony benchranks

company

aboutvisionmissionteaminstructorsteach on explainxpartnershipscommunityhackathonscareers

content

daily AI newsstate of AI — live resultsblogreleasespromptsgeneratorsresource libraryfor LLMsexplainx.ai kids

solutions

all solutionsdeveloper upskillingmarketing upskillingproduct manager upskillingleadership upskilling

newsletter · weekly

Get AI news, tools, and insights in your inbox.

supportcontactprivacytermsdata rightshow we create contentsubmission guidelines

© 2026 AISOLO Technologies Pvt Ltd

explainx.ai

On this page

  • TL;DR
  • What vLLM actually published
  • Why 93.3× spreads anyway
  • How to verify before you quote a multiplier
  • What infra engineers ask about the Rubin charts
  • Kernel work behind the multiplier (simplified)
  • Cost per million tokens still matters
  • GB300 baseline numbers vLLM published earlier
  • Step-by-step: verify a viral inference chart
  • Historical pattern: Kimi K3 and vLLM headlines
  • When 93.3× might have appeared (hypothesis)
  • Open-source inference stack implications
  • Workshop tie-in for platform engineers
  • Dynamo, SGLang, and other engines on the same silicon
  • Power-normalized throughput tab
  • Failure modes when quoting Rubin
  • Builder takeaway
  • Related reading
  • Glossary for the Rubin blog post
  • Capacity planning worksheet (conceptual)
  • If you are writing a press release
← Back to blog

explainx / blog

vLLM Rubin 93.3× Throughput Claim: What Social Posts Say vs vLLM Blog

vLLM, Inference, Nvidia, DeepSeek, Benchmarks

Part of AI Chips and Infrastructure

Social posts cite 93.3× GB300 throughput for DeepSeek on Rubin. vLLM official November 12 blog documents 58× at 387 tokens/s per user on Vera Rubin NVL72 — here is how to reconcile the numbers.

Nov 17, 2026·8 min read·Yash Thakker
add explainx.ai
go deep
vLLM Rubin 93.3× Throughput Claim: What Social Posts Say vs vLLM Blog

Social feeds this week repeated that vLLM achieved 93.3× the token throughput of GB300 for DeepSeek inference on NVIDIA Rubin — but the vLLM project’s own November 12, 2026 technical post documents a 58× gain at a defined 387 tokens/s per user operating point. For builders sizing agentic inference, the gap between viral multipliers and chart footnotes is the story — not a single headline integer.

Our Kimi K3 vLLM claimed-vs-verified post showed the same pattern: reposts round up; primary sources anchor concurrency, precision, and token definitions.

TL;DR

table · 3 cols
SourceMultiplier citedConditions stated
vLLM blog (primary)58× vs GB300 NVL72387 tok/s/user P90, AgentX, P/D + Wide-EP
Viral posts93.3×Often no matching footnote in explainx.ai checks
SemiAnalysis / InferenceX tables11×–21×+Different tok/s/user targets on same model
Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

What vLLM actually published

SemiAnalysis AgentX throughput on Vera Rubin NVL72 versus GB300 NVL72SemiAnalysis AgentX throughput on Vera Rubin NVL72 versus GB300 NVL72 Image: vLLM

vLLM’s Rubin write-up is the third DeepSeek-V4.1-Flash installment:

  1. Day-zero kernels and fusions
  2. GB300 rack scaling adjacency — see also October GB300 PD post
  3. Rubin NVL72 move with SM107-tuned kernels and rack-wide KV cache

Headline numbers from the primary post:

  • ~1.7M tokens/s/GPU at 387 tok/s/user — 58× GB300 at that point
  • 2.52M tokens/s/GPU peak in their throughput sweep
  • ~1,810 tok/s/user on a specialized TP decode path with speculative decoding stats

The post stresses counting methodology: totals include cached input, uncached input, and output tokens, aligned to the earlier GB300 chart for comparability.

Why 93.3× spreads anyway

Aggregators and X accounts often:

  1. Cherry-pick a peak chart point without the per-user latency constraint your product promises.
  2. Mix vendors — SemiAnalysis public compare tools plot different tok/s/user targets (see InferenceX GB300 vs Vera Rubin compare).
  3. Confuse per-GPU vs per-MW tabs on the same figure.

None of that makes Rubin uninteresting — NVIDIA’s co-design story and CoreWeave’s Vera cluster still matter — but procurement math needs your SLO.

How to verify before you quote a multiplier

  1. Open the vLLM figure caption and note tok/s/user, precision, and engine (vLLM vs Dynamo).
  2. Match AgentX vs fixed-sequence if you serve chat not coding agents.
  3. Compare cost per million tokens only when power and lease assumptions match — InferenceX adds $/M tok columns.
  4. Cross-read Baseten’s efficient frontier guide for serving economics framing.

What infra engineers ask about the Rubin charts

Why does vLLM use AgentX instead of ShareGPT? AgentX replays coding-agent traces with long caches — matching DeepSeek V4.1 Flash serving economics more than short chat bursts.

What is P/D disaggregation? Prefill and decode run on different workers so bursty prompt phases do not starve token generation — vLLM used the same pattern on GB300.

What is Wide-EP? Wide expert parallelism spreads MoE experts across many GPUs. DeepSeek-class models need it at rack scale.

Can I reproduce 58× on one B300? No. The claim is rack vs rack: Vera Rubin NVL72 against GB300 NVL72.

Where did 93.3× appear? explainx.ai saw it on reposts, not in the vLLM blog paragraph that states 58×. Some third-party compare pages show other ratios at 567 tok/s/user — always read the x-axis.

Should startups wait for Rubin? If GB300 contracts are cheap today, 58× is not a reason to idle; it is a reason to renegotiate when Rubin racks hit your cloud’s catalog.

Kernel work behind the multiplier (simplified)

vLLM’s Rubin post highlights:

  • SM107 FP8×FP4 MMA paths tuned for Rubin tensor cores
  • DEP2 prefill and DEP decode kernel iterations that improved 28% and 25% vs their first Rubin bring-up
  • Mooncake KV pool plus Engram sharding over NVLink holding 96.5–97% prefix hit rates at high concurrency

Those details explain why unsourced multipliers fail: throughput is not a single chip spec — it is kernels + cache routing + concurrency target.

Cost per million tokens still matters

GPU cluster rack illustrating rack-scale inference economicsGPU cluster rack illustrating rack-scale inference economics

SemiAnalysis-derived tables (via InferenceX) pair throughput with $/M tokens at named tok/s/user points. A 93.3× throughput repost with no price hides whether Rubin saves money after power and lease.

Pair this post with NVIDIA five co-design tactics when CFOs ask per-megawatt not per-GPU.

GB300 baseline numbers vLLM published earlier

To understand why 58× is already extreme, recall vLLM’s October 23, 2026 GB300 post:

  • 954K tokens/s/GPU at 196 tok/s/user P90 on AgentX with P/D and Wide-EP
  • 826 P90 tok/s/user on a specialized single-user decode path on four B300 GPUs
  • Claimed within 6% decode throughput of DeepInfer at batch 8–256

The Rubin post’s 1.7M tokens/s/GPU at 387 tok/s/user is not a free lunch — it is a different operating point on the same benchmark family. Comparing multipliers without naming that point is how 93.3× memes spread.

Step-by-step: verify a viral inference chart

  1. Screenshot the chart and find the footnote on tok/s/user.
  2. Open the linked vLLM GitHub PR or blog — confirm engine commit.
  3. Check whether cached tokens count toward numerator (vLLM says yes for comparability).
  4. If a tweet cites SemiAnalysis, open the InferenceX URL and note interpolation disclaimers.
  5. Only then quote a ratio in a board deck.

Historical pattern: Kimi K3 and vLLM headlines

Our Kimi K3 842 tok/s post walked the same beat: social posts rounded community benchmarks into product claims. The fix is never “ignore Rubin” — it is attribute numbers or write claimed-vs-verified honestly.

When 93.3× might have appeared (hypothesis)

explainx.ai did not find 93.3× in vLLM primary text. Possibilities unconfirmed:

  • Misread chart axis (per-MW vs per-GPU tab)
  • Compound ratio (multiple improvements multiplied)
  • Typo for 58× or 83.3× in a repost chain

Until a primary source says 93.3× with methodology, do not capacity-plan on it.

Open-source inference stack implications

vLLM publishing Rubin numbers helps self-hosters on Hugging Face OpenRouter routing debates — proof that open engines chase rack-tuned vendor stacks. If you run SGLang instead, read sibling posts on MMFlash speculative decoding before assuming identical curves.

Workshop tie-in for platform engineers

Teams operating multi-tenant inference should rehearse how they communicate benchmarks to customers. Mis-stated 93.3× claims become SLA disputes. Document measurement scripts alongside marketing.

Dynamo, SGLang, and other engines on the same silicon

InferenceX tables list Dynamo vLLM, Dynamo SGLang, SGLang, TileRT, and plain vLLM on GB300 runs. The Rubin vLLM article is one engine story — your cloud might ship Dynamo wrappers with different caching. When a headline says vLLM beat DeepInfer by 6%, that is vendor engineering pride, not a guarantee your managed endpoint exposes the same binary.

Teams debating Hugging Face vs OpenRouter routing should ask providers which chart they use in capacity planning — not which tweet they retweeted.

Power-normalized throughput tab

vLLM’s Figure 1 includes a per-megawatt tab — crucial when data-center moratorium politics from OpenAI Grid Protectors and Philippines protest math dominate permitting. A 58× per-GPU win that costs 2× power may still lose on tokens per megawatt.

Failure modes when quoting Rubin

table · 2 cols
MistakeConsequence
Quote peak 2.52M without concurrencyCustomer SLA miss on tail latency
Ignore cached token accountingApples-to-oranges vs competitor blog
Use 93.3× from XProcurement based on fiction
Skip preview disclaimers on Rubin hardwareChart may change before GA

Builder takeaway

Rubin plus vLLM is a real stack upgrade for trace-heavy agent workloads, but 93.3× is not on the vLLM blog explainx.ai fetched. Use 58× at 387 tok/s/user as the documented apples-to-apples claim until vLLM or SemiAnalysis publishes a footnote that explicitly states 93.3× with the same denominator.

Related reading

  • Kimi K3 842 tok/s vLLM claimed vs verified
  • DeepSeek DSpark speculative decoding
  • NVIDIA Rubin NVL72 SemiAnalysis throughput
  • Hugging Face OpenRouter routing update
  • Baseten efficient frontier LLM inference
  • DeepSeek free chat compute limits
  • Together AI Ollama prefix cache for long agents
  • Moonshot coding benchmark flattening

Glossary for the Rubin blog post

P90 tok/s/user — ninetieth percentile of per-user generation speed across the AgentX trace mix; higher means snappier UX at a given concurrency.

Wide-EP — spreads mixture-of-experts layers across many GPU ranks so no single card holds all experts.

Mooncake KV pool — shared prefix cache architecture vLLM cites for rack-scale hits above ninety-six percent.

DEP2 kernels — vLLM named prefill/decode kernel iterations tuned for Rubin SM107; the blog cites double-digit percent gains over their first Rubin port.

DeepInfer — DeepSeek’s referenced inference engine; vLLM compares decode throughput to stay within six percent on GB300.

Capacity planning worksheet (conceptual)

table · 2 cols
InputYour value
Target tok/s/user for chat UXFill from product requirements
AgentX-like cache hit rateAsk provider or measure traces
Rack typeGB300 vs Vera Rubin NVL72
EnginevLLM version + commit
Claimed multiplier from socialVerify against vLLM footnote

Fill the worksheet before signing Rubin preorder contracts tied to marketing decks.

If you are writing a press release

Quote 58× only with the 387 tok/s/user footnote, link vLLM.ai, and disclose preview hardware. Anything else invites claimed-vs-verified corrections within hours on social feeds.

Accurate as of November 17, 2026. Throughput ratios follow vLLM.ai posts dated October–November 2026 unless labeled as unverified social claims.

Spotted something out of date? Let us know.
Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

  • ✓ Free
  • ✓ One email a day, no spam
  • ✓ Unsubscribe anytime
Yash Thakker

Written by

Yash Thakker

Yash is an AI expert with over 300K learners. Join his workshops →

View Yash Thakker in People in AI →

Related posts

Nov 10, 2026

Naive-N0.5-Flash: 2,100 tok/s Open Model and Latent Space Coding Test

NaiveAI markets Naive-N0.5-Flash as an open 309B MoE coding model with NaiveRT inference up to roughly 2,000 tokens per second. On November 10, 2026 Latent Space published an unbiased hands-on versus DeepSeek V4.1 Flash 1017 and GLM 5.3 Flash. We separate NaiveAI's peak-speed claims from end-to-end latency and app quality.

Nov 14, 2026

192GB Local AI Without a $9,500 Mac: Hardware Paths and Real Trade-offs

You need about 192GB of fast memory to run quantized DeepSeek-class models with any prompt headroom. This guide maps the non-Mac options Martin Weinberg demos on YouTube and cross-checks them against Lucebox and Framework public benchmarks.

Nov 14, 2026

Luminal Series A: Search Compiler Targets Multi-Chip LLM Inference

On November 12, 2026 Luminal announced a $25 million Series A led by Standard Capital for a search compiler that maps models to GPUs and ASICs before serving traffic. The startup claims 36,000 tokens per second on GPT-OSS 120B on eight H100s versus 26,000 for vLLM. Here is how the compiler story fits next to Rust engines like Baseten DeepSeek Ultrafast.