NVIDIA shipped the first model in the Nemotron 3.5 family on August 11, 2026: Nemotron 3.5 Lightning, a 30-billion-parameter Mixture-of-Experts model with only 3 billion active parameters, a 1M-token context window, and weights under the permissive OpenMDW-1.1 license. It is the successor to Nemotron 3 Nano 30B and, per OpenRouter's listing, a distillation of the much larger Nemotron 3 Ultra, the 550B open MoE NVIDIA shipped in June.
The pitch from NVIDIA AI is narrow and honest: this is not a frontier model. It is "built for always-on agents to complete high-volume, specialized tasks faster," delivering "up to 4x the output speed of similar-sized models." That framing matters, because the benchmark table underneath it only makes sense if you read speed as the product and intelligence as the constraint.
It went live the same day on Together AI, DeepInfra, Fireworks, and OpenRouter.
NVIDIA's own launch chart. Note what it concedes: Qwen3.6-35B sits marginally higher on PinchBench accuracy — Nemotron's win is the horizontal axis, not the vertical one.
TL;DR — what people are asking
| Question | Answer |
|---|---|
| What is it? | 30B total / 3B active MoE, hybrid Mamba-2 + Attention + MoE |
| Who made it? | NVIDIA, distilled from Nemotron 3 Ultra |
| License? | OpenMDW-1.1 — permissive, commercial use explicitly allowed |
| Context window? | Up to 1,000,000 tokens |
| Minimum hardware? | One GPU — DGX Spark (GB10) or H100; Ampere via W4A16 |
| Is it fast? | NVIDIA claims up to 4x output speed; OpenRouter claims up to 30% faster task completion |
| Is it smart? | 24 on the Artificial Analysis Intelligence Index — roughly gpt-oss-120b at a quarter the parameters |
| Good for coding agents? | As a sub-agent, yes. As your primary agent, no — Terminal-Bench 2.1 is 24.58 |
| Where do I get it? | Hugging Face, plus Together AI, DeepInfra, Fireworks, OpenRouter |
| Is training data released? | Yes — final pre-training and post-training data, mostly gated |
What "30B-A3B" actually buys you
The suffix is the whole design. 30B total parameters, 3B active per token means the router selects a small fraction of the expert weights for any given forward pass. You pay 30B of memory and 3B of compute.
Stack that on top of a hybrid Mamba-2 backbone and the economics get more interesting. Standard Transformer attention grows its KV cache linearly with sequence length, which is why long-context serving usually collapses under concurrency long before it collapses under context. Mamba layers carry a fixed-size recurrent state instead — so most of the 1M-token window costs constant memory rather than linear memory. Only the "select Attention layers" NVIDIA interleaves into the stack pay full freight.
This is the same architectural bet explainx.ai tracked in Kimi K3's LatentMoE and NoPE design and in NVIDIA's own Nemotron-Labs-TwoTower diffusion experiment, which was built on the previous-generation Nemotron-3-Nano-30B-A3B. The Nemotron line has been converging on "sparse activation plus linear-state sequence mixing" for a year; Lightning is the production version of that convergence.
| Spec | Value |
|---|---|
| Total parameters | 30B |
| Active parameters | 3B |
| Architecture | Mamba-2 + MoE + Attention hybrid |
| Pre-training tokens | More than 20 trillion |
| Pre-training precision | NVFP4 recipe |
| Context length | Up to 1M tokens |
| Spoken languages | English, Spanish, French, German, Italian, Japanese |
| Programming languages | 43 in pre-training |
| Recommended sampling | Temperature 1.0, Top_P 0.95 |
| Data cutoff | Pre-training Sept 2025; post-training May 2026 |
The benchmarks, read honestly
NVIDIA published results for both the BF16 model and the NVFP4 checkpoint it actually released, measured "under a consistent harness (NeMo Gym / Nemo Evaluator SDK)" — with the explicit caveat that they "may differ from vendors' self-reported numbers." Evaluation recipes, containers, prompts, and scoring configs were published in NeMo Gym for reproduction, which is more transparency than most launch tables come with. (For what these suites measure, see explainx.ai's AI benchmarks guide.)
| Benchmark | BF16 | NVFP4 (released) |
|---|---|---|
| MMLU Pro | 81.94 | 81.62 |
| AA-Omniscience | 17.50 | 16.63 |
| GPQA Diamond (no tools) | 75.44 | 75.57 |
| HLE (text-only, no tools) | 11.72 | 10.47 |
| SciCode | 32.60 | 31.38 |
| SWE-bench Verified | 51.56 | 52.80 |
| SWE-bench Multilingual | 39.33 | 36.47 |
| Terminal-Bench 2.1 | 24.58 | 23.46 |
| PinchBench | 85.37 | 83.43 |
| BrowseComp | 36.97 | 36.81 |
| tau-cubed-bench (Banking) | 9.28 | 9.48 |
| GDPval-AA-V2 | 832 | 865 |
| IFBench (loose) | 71.88 | 72.88 |
| AA-LCR | 52.00 | 49.19 |
Three things this table says that the announcement thread does not.
1. MMLU Pro at 81.94 is genuinely strong for 3B active parameters. So is GPQA Diamond at 75.44. On knowledge and single-shot reasoning, this model punches far above its activation budget — which is exactly what Artificial Analysis meant by "similar to gpt-oss-120b at around a quarter of the total parameters."
2. The agentic long-horizon scores are weak. Terminal-Bench 2.1 at 24.58 and tau-cubed-bench Banking at 9.28 are not competitive numbers in absolute terms. For scale: Meta's Muse Glimmer 30B, released the day before at a comparable size, posts 51.7 on Terminal-Bench 2.1 and 23.5 on tau-cubed Banking. A model that scores 85 on PinchBench and 24 on Terminal-Bench is telling you something specific: it executes well-scoped steps reliably and falls apart over long autonomous horizons.
3. Several NVFP4 scores go up after quantization. SWE-bench Verified rises 51.56 to 52.80, GDPval-AA-V2 rises 832 to 865, IFBench rises 71.88 to 72.88. Quantization does not make a model smarter. These are run-to-run variance, and their size is a useful calibration for how much any single number on this table is worth — roughly plus or minus a point on percentage-scale benchmarks. The real quantization damage shows up where the drops exceed that band: AA-LCR (-2.81) and SWE-bench Multilingual (-2.86), both long-context or multilingual tasks.
About that "35% faster" number
The social summaries circulating around launch cited GDPval-AA v2 at 824 Elo, 86% on PinchBench, and 10,000 tasks finished 35% faster. The model card's own figures are 832/865 on GDPval-AA-V2 and 85.37/83.43 on PinchBench — close, but the rounded social versions drifted upward.
For the speed claim, the numbers with sourcing behind them are NVIDIA's "up to 4x the output speed of similar-sized models" (tokens per second) and OpenRouter's "up to 30% faster task completion" (wall-clock to finish an agent trajectory). Those measure different things and both carry "up to." NVIDIA's own chart plots ~17 H100 GPU-hours for 10,000 tasks against roughly 24 for Qwen3.6-35B and 25.5 for Gemma 4 26B — call it 29-33%, consistent with OpenRouter's figure.
Deploying it: one GPU, three drafters
The released checkpoint is nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4. It weighs about 18B parameters on disk because the published weights are already post-training quantized — NVIDIA did not ship a BF16 checkpoint and a separate quantized one; the quantized model is the product. The PTQ recipe is "Four Over Six NVFP4" (a static MSE calibration variant), W4A16 on routed and shared experts, FP8 per-tensor dynamic scales on the Mamba in_proj/out_proj and the KV cache, calibrated on 1,000 samples at 32K token length.
Speculative decoding options
Lightning ships with three ways to generate more than one token per forward pass — the same lever explainx.ai covered in NVIDIA's FastGen parallel-decoding distillation work:
| Method | What it does | Use when |
|---|---|---|
| DSpark | Semi-autoregressive drafter proposing a whole token block from a parallel backbone | DGX Spark and low-concurrency data-centre serving — NVIDIA's current default recommendation |
| DFlash | Lightweight block-diffusion drafter generating a draft block in one forward pass | Alternative drafter; ships as a separate checkpoint |
| MTP | Multi-Token Prediction heads trained into the model itself | No separate drafter to load; also used to accelerate RL rollouts during training |
Single-GPU serving on H100
Max-throughput configuration — NVIDIA notes that no speculative decoding is best here, and that memory constraints push the Mamba cache to FP16:
export MODEL_CKPT=nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4
vllm serve --model $MODEL_CKPT \
--max-num-seqs 256 \
--max-num-batched-tokens 16384 \
--enable-prefix-caching \
--async-scheduling \
--mamba-backend flashinfer \
--moe-backend humming \
--linear-backend humming \
--mamba-ssu-algorithm horizontal \
--mamba-cache-mode align \
--mamba-ssm-cache-dtype float16 \
--enable-mamba-cache-stochastic-rounding \
--mamba-cache-philox-rounds 5 \
--reasoning-parser nemotron_v3 \
--tool-call-parser qwen3_coder \
--enable-auto-tool-choice
For interactive workloads — NVIDIA targets 40+ tokens/sec per user — drop concurrency to 128 or fewer and switch DSpark on:
export DSPARK_CKPT=nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark
vllm serve --model $MODEL_CKPT \
--max-num-seqs 128 \
--enable-prefix-caching \
--async-scheduling \
--speculative_config.model $DSPARK_CKPT \
--speculative_config.num_speculative_tokens 3 \
--mamba-ssu-algorithm horizontal \
--mamba-backend flashinfer \
--mamba-ssm-cache-dtype float16 \
--reasoning-parser nemotron_v3 \
--tool-call-parser qwen3_coder \
--enable-auto-tool-choice
On a DGX Spark
The GB10 recipe is the one NVIDIA leads its model card with, and it is the most consequential line in the release for anyone who bought the box explainx.ai reviewed in its DGX Spark local LLM guide — a 1M-context agentic model running on a single desktop-class machine:
vllm serve --model $MODEL_CKPT \
--moe-backend marlin \
--kv-cache-dtype fp8 \
--max-model-len 1048576 \
--enable-prefix-caching \
--speculative_config.method dspark \
--speculative_config.model $DSPARK_CKPT \
--speculative_config.num_speculative_tokens 3 \
--mamba-backend flashinfer \
--mamba-cache-mode align \
--reasoning-parser nemotron_v3 \
--tool-call-parser qwen3_coder \
--enable-auto-tool-choice
TensorRT-LLM (nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc24) and SGLang (lmsysorg/sglang:dev-nemotron3-5-lighting) recipes are both published on the model card. Note the full 1M context is served by default on the H100 and GB200 snippets — if you are memory-constrained or want KV headroom at high concurrency, lower --max-model-len rather than discovering the limit under load.
Tool calling gotcha
The tool-call parser is qwen3_coder and the reasoning parser is nemotron_v3. One non-obvious requirement from the model card: for coding agents, you must pass force_nonempty_content, or you will get tool calls with empty message content that many harnesses reject.
response = client.chat.completions.create(
model=MODEL,
messages=[{"role": "user", "content": "What's the weather in Santa Clara?"}],
tools=tools,
max_tokens=16000,
temperature=1.0,
top_p=0.95,
extra_body={"chat_template_kwargs": {"force_nonempty_content": True}},
)
Reasoning is on by default and can be disabled per-request through chat-template kwargs — useful if you are routing simple sub-agent calls through the same endpoint and do not want to pay for thinking tokens on every one. If you are wiring this into a tool-using stack, explainx.ai's MCP guide covers the protocol side of the same problem.
How it was trained — and one detail worth noting
NVIDIA describes a five-stage pipeline:
- Pre-training — over 20T tokens of crawled and synthetic code, math, science, and general knowledge, using an NVFP4 recipe in Megatron-LM.
- Continued pre-training for MTP — training the Multi-Token Prediction heads to align with the base model's distribution.
- Supervised fine-tuning — synthetic code, math, science, tool calling, instruction following, and structured outputs, plus data built for long-range retrieval and multi-document aggregation.
- Reinforcement learning — multi-environment GRPO across math, code, science, instruction following, multi-step tool use, multi-turn conversation, and structured output, using an asynchronous architecture that decouples training from inference and uses MTP to accelerate rollout generation.
- Post-training quantization — the NVFP4 recipe described above.
Two details stand out.
The teacher for synthetic data was gpt-oss-120b. NVIDIA states that for math, code, and science it used "open source permissive models such as GPT-OSS-120B to produce step-by-step reasoning traces, candidate solutions, best-of-n selection traces, and verified CUDA kernels." An open-weight model bootstrapping the next open-weight model, with no closed-model license entanglement — the same distillation economics explainx.ai examined in Intology's automated post-training work.
NVIDIA filtered political alignment out of teacher outputs. From the model card: internal audits found "some teacher models occasionally produce reasoning traces and final responses that implicitly align with specific political entities or promote nationalistic narratives," so NVIDIA applied "targeted keyword- and regex-based filters and remove[d] all trajectories matching such behavior." That is an unusually specific disclosure, and a real one — it is a documented failure mode of synthetic-data pipelines that most labs do not mention.
NVIDIA also released the final pre-training and post-training data, with an ungated sample set and gating plus approval for the remaining code, math, and multilingual data. That puts the release closer to genuinely open than the "weights-only" norm — the distinction explainx.ai unpacked in open-weight vs closed models.
The license: OpenMDW-1.1, not Apache 2.0
Governing terms are the OpenMDW License Agreement, version 1.1 — a model-specific permissive license covering weights, data, and supporting artifacts together, rather than a general software license retrofitted onto a checkpoint. NVIDIA's model card states directly: "This model is ready for commercial use."
For most teams that is the end of the analysis: you can serve it, fine-tune it, and ship products on it. But it is not an OSI-approved open-source license, and it is not Apache 2.0. If you are in a regulated environment where legal signs off on license families rather than individual terms, read it rather than assuming equivalence — the same due-diligence gap that shows up across the open-weight releases in explainx.ai's laptop-runnable model roundup.
Where this model is the wrong choice
Derivative coverage that skips this section is not useful. Based on NVIDIA's own numbers:
- Do not make it your primary coding agent. 24.58 on Terminal-Bench 2.1 means long autonomous terminal work will stall. Route planning to a stronger model and use Lightning for the fan-out.
- Do not use it for complex multi-turn tool workflows. 9.28 on tau-cubed-bench Banking is a low score on a benchmark specifically about sustained, policy-constrained tool use.
- Do not assume the 1M context is free. It is served by default in NVIDIA's snippets, but KV headroom at high concurrency is the first thing to break. Mamba layers keep most of it constant-cost; the interleaved attention layers do not.
- Do not read "24 on the AA Intelligence Index" as frontier-adjacent. It is a strong number per active parameter. It is not a strong number in absolute terms, and NVIDIA has not claimed otherwise.
- Do not benchmark-shop on the quantization deltas. Several NVFP4 scores exceeding BF16 is noise; treat differences under about a point as meaningless.
Where it is right: high-volume sub-agent pipelines, the worker tier in a planner-worker-critic topology, retrieval and summarization over long documents, and specialized post-training on your own domain data — NVIDIA explicitly pitches coding, legal, and cybersecurity fine-tunes. That maps cleanly onto the delegation patterns in explainx.ai's agent harness guide and loop engineering primer, where the expensive model plans and cheap models execute.
What about Nemotron 4?
A widely-shared post from a finance account claimed NVIDIA is developing Nemotron 4, a 1T+ parameter open-source model. This is unverified. It did not come from NVIDIA, there is no model card, no announcement, and no date. Treat it as market commentary rather than a roadmap.
What is on the record: Lightning is explicitly "the first Nemotron 3.5 model," which means more of the family is coming. NVIDIA's stated strategy — open models that broaden AI adoption and therefore GPU demand — has been consistent enough across Nemotron Labs and its research releases that continued open-weighting is a safe bet. The parameter count is not.
Related on explainx.ai
- NVIDIA Nemotron 3 Ultra: the 550B open MoE this model was distilled from
- NVIDIA Nemotron-Labs-TwoTower: splitting a 30B model for 2.42x faster generation
- NVIDIA DGX Spark: the best setup for running local LLMs
- Meta Muse Glimmer: a 30B open-weight agentic model
- Kimi K3 architecture: LatentMoE and NoPE
- Tencent HY3: a 295B open-source agentic MoE
- Cursor's Mixture-of-Kittens MoE megakernel
- FastGen: NVIDIA's parallel decoding distillation
- Top 10 open-weight models you can run on a laptop
- How to choose open-weight vs closed AI models
- What is an agent harness? Complete guide
Official source: NVIDIA, NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 model card on Hugging Face (August 11, 2026).
Specs, benchmark figures, and serving commands reflect NVIDIA's Hugging Face model card as published on August 11, 2026, plus launch-day posts from NVIDIA AI, OpenRouter, and Artificial Analysis. All accuracy numbers are NVIDIA's own, measured under its NeMo Gym / Nemo Evaluator harness, and NVIDIA notes they may differ from vendors' self-reported results — verify on your own workload before routing production traffic. The chart above is NVIDIA's own launch graphic. Provider availability, pricing, and vLLM/TensorRT-LLM/SGLang flags change quickly; check the model card for current status.