NVIDIA has released Muse Spark Open DFlash 2, a 4.38B-parameter draft model that pairs with Meta's 816B open-weight Muse Spark Open to run speculative decoding. According to its Hugging Face model card, published October 16, 2026, the draft accepts an average of 3.73 tokens per step on MT-Bench at greedy decoding, and about 3.01 when sampling at temperature 1.0.
The short version: this is a speed add-on for a model that was already hard to serve, not a new model you chat with. It matters if you operate Muse Spark Open yourself, because decode speed is the main cost lever for long agentic coding sessions. It does not matter much if you only call the model through a hosted API.
TL;DR: the questions people will ask
| Question | Answer |
|---|---|
| What is it? | A DFlash 2 draft checkpoint for Muse-Spark-Open-816B-A42B, trained by NVIDIA |
| Size? | 4.38B parameters, dense, 5 decoder layers; embeddings and output head are shared with the target |
| Who made the target? | Meta. NVIDIA says it does not own the target model |
| Headline number? | Acceptance length 3.73 (MT-Bench, greedy, draft length 7) |
| With sampling? | About 3.01 at temperature 1.0 |
| Worst case in the card? | 2.16 on high-entropy prompts with sampling, 2.51 greedy |
| Hardware and runtimes? | NVIDIA Blackwell, Linux, vLLM and TRT-LLM; tested on a GB300 node |
| Is there a measured tokens-per-second gain? | Not on the card. Acceptance length only |
| Licence? | NVIDIA Open Model Agreement plus Meta's Muse Model License |
What is speculative decoding, in plain terms?
Fast draft step followed by a slower verification step with a green check, illustrating speculative decoding
A large language model normally produces one token per forward pass. For an 816B mixture-of-experts model that is a slow, memory-bound loop: every step reads a lot of weights to emit a single token.
Speculative decoding adds a small draft model. The draft guesses the next several tokens cheaply. The big target model then checks all of those guesses in one pass and keeps the longest correct prefix. If the draft is right about four tokens, you got four tokens for roughly the price of one target pass. The mean number of tokens emitted per step is the acceptance length, and it is the number NVIDIA reports.
Because the target verifies the guesses, a weak draft costs you speed but not correctness. That is why the technique is popular for serving: it is the closest thing inference has to a free lunch. We cover the same idea for other stacks in DeepSeek's DSpark guide and in our explainer on what tokens per second really measures.
What is DFlash, and what does DFlash 2 add?
DFlash comes from the paper DFlash: Block Diffusion for Flash Speculative Decoding by Jian Chen, Yesheng Liang and Zhijian Liu of Z Lab, posted February 5, 2026. Its idea: instead of drafting tokens one after another, use a lightweight block diffusion model that proposes a whole block of tokens in a single forward pass, conditioned on features taken from the target model. The paper reports over 6x lossless acceleration across a range of models and tasks and up to 2.5x higher speedup than EAGLE-3, a leading autoregressive drafting method. Those are the authors' numbers on their own setups, not on Muse Spark Open.
NVIDIA's card describes the new variant like this: DFlash 2 "adds a two-tap in-block convolution around every sublayer and a pairwise path selector that walks one coherent path through each position's top-k candidates." In plain terms, two changes:
- Local mixing inside the block. The convolution lets neighbouring draft positions see each other's information, which helps the draft stay consistent across a block it produces all at once.
- A path selector. A parallel drafter proposes several candidates per position. Picking each position independently can stitch together a sequence that no coherent text would produce. The pairwise selector chooses one connected path through those candidates, so the verifier is handed a more plausible chain and accepts more of it.
NVIDIA says the draft was built with its Model Optimizer library, specifically a pre-release build of version 0.47, and ships in BF16.
An earlier community port of the first DFlash is already on Apple hardware; our write-up of dflash-mlx on an M5 Max shows how widely the numbers swing by model and context length, a caution that applies here too.
What are the model specs?
Pulled from the model card:
- Architecture: a dense transformer of 5 decoder layers using non-causal sliding-window attention with a window of 2,048 tokens.
- Parameters: 4.38B total and active. The token embedding and output head are shared with the target and not counted.
- Vocabulary: 202,048, the same as Muse Spark Open.
- Context: combined input and output of up to 1,000,000 tokens, matching the target.
- Inputs: text, image, video and audio are listed, since it follows the multimodal target; output is text.
- Training data: prompts from NVIDIA's Nemotron-Post-Training-Dataset-v2. The original responses were not used. Instead the target model generated synthetic responses at five reasoning-effort levels, including reasoning traces, giving about 3.48 million multilingual samples across math, code, STEM and chat.
- Precision: BF16.
Training on the target's own outputs is the standard trick: the draft learns to imitate exactly the distribution it will later be checked against, which is what pushes acceptance up.
How do you run it?
Open-weights release illustration for serving a large model with a companion draft checkpoint
The card gives a single vLLM recipe:
vllm serve meta-models/Muse-Spark-Open-816B-A42B \
--speculative-config '{"method": "dflash", "model": "nvidia/Muse-Spark-Open-816B-A42B-DFlash2", "num_speculative_tokens": 7, "draft_sample_method": "probabilistic"}' \
--trust-remote-code
Details worth noting:
- Seven speculative tokens is the draft length used for every table. Acceptance length cannot exceed the draft length plus the bonus token the target emits, so 3.73 out of a possible 7 means a little over half the drafted block survives on average.
draft_sample_method: probabilisticis what the sampling (t=1) numbers used. The greedy numbers use greedy decoding.- The target is gated. Meta's weights sit behind a licence form on Hugging Face, so you need access to the target before this command works. The licence terms and the Model-as-a-Service threshold are broken down in our Muse Spark Open guide.
- Hardware: the card lists NVIDIA Blackwell, Linux, and vLLM or TRT-LLM. All measurements used one GB300 node with tensor parallel 4 and expert parallel 4.
NVIDIA's earlier technical post on running the model on GB300 NVL72, which we summarised in the main guide, quoted up to 600 tokens per second per user with speculative decoding. That is a vendor figure for a rack-scale system; this card does not break it down by draft model.
What do the acceptance-length numbers say?
All measurements used reasoning effort xhigh, 4,096 output tokens, concurrency 32, draft length 7, and the NVFP4 target checkpoint, according to the card.
| Benchmark | Greedy (t=0) | Sampling (t=1) |
|---|---|---|
| MT-Bench, overall | 3.73 | 3.01 |
| SPEED-Bench qualitative, overall | 3.72 | 3.01 |
| SPEED-Bench throughput, 32k-token inputs | 3.30 | 2.78 |
By category, the pattern is consistent:
| Category (MT-Bench, greedy) | Acceptance length |
|---|---|
| Extraction | 4.60 |
| Math | 4.47 |
| Coding | 4.34 |
| Reasoning | 3.52 |
| Writing | 3.60 |
| Humanities | 2.92 |
On SPEED-Bench the best greedy categories are multilingual (4.27), RAG (4.17) and coding (4.15), and the weakest are writing (3.05) and humanities (3.23). On the long-input throughput split, low-entropy prompts reach 3.71 greedy while high-entropy prompts fall to 2.51, and to 2.16 with sampling.
How to read this
- Predictable text drafts well. Code, extraction and math have strong structure, so the draft guesses right more often. Open-ended writing has many valid continuations, so fewer guesses match. For an agentic coding model, which is Muse Spark Open's stated purpose, the 4.15 to 4.34 coding figures are the relevant ones.
- Sampling costs acceptance. Moving from greedy to temperature 1.0 drops overall acceptance from about 3.7 to 3.0. Most chat and agent traffic samples rather than decoding greedily, so the sampled column is usually closer to real use.
- Acceptance length is not speedup. If the draft takes 15 percent of a target step's time and accepts 3.0 tokens, the gain is well under 3x. The card publishes no tokens-per-second comparison with and without the draft, so any "N times faster" claim about this checkpoint is not supported by the source. Measure on your own traffic.
- Concurrency matters. The tests ran at concurrency 32. At high batch sizes the target is closer to compute-bound and speculative decoding helps less than at low concurrency. Single-user latency is where drafts shine most.
How does this compare with other draft approaches?
| Approach | How it drafts | Example on explainx.ai |
|---|---|---|
| Autoregressive small model | One token at a time from a small LM | Classic speculative decoding |
| Multi-token prediction heads | Extra heads on the target | Ollama 0.31 and Gemma 4 MTP |
| Block diffusion drafter | A whole block in one pass | DFlash and DFlash 2 |
| Tiny trained drafter inside a compressed model | Small drafter tuned alongside quantization | Nemotron 3.5 Bolt with a 2-bit drafter |
The trend is clear: drafting is moving from sequential to parallel, because sequential drafting becomes its own bottleneck once the draft has to guess many tokens ahead. NVIDIA is also working on the other side of the stack, as in its ReplaySSM research on hybrid decoding, and engines such as TokenSpeed target agentic workloads from the runtime side.
Why would NVIDIA ship a draft for Meta's model?
Two reasons are visible in the sources. First, the card says the Muse Spark Open release is "intended for DFlash 2-assisted serving", which makes the draft part of NVIDIA's serving recipe for Blackwell hardware. A model that decodes faster per user makes a GB300 node more useful per dollar. Second, it is another sign of how the open-weight ecosystem now works: Meta ships the weights, then NVIDIA, Apple, AWS and community projects ship the pieces that make them practical, such as Apple's four-Mac-Studio MLX cluster demo. Meta's smaller sibling, Muse Glimmer, already shipped with its own DFlash drafter, which is the same pattern at a smaller scale.
Limitations and open questions
- No end-to-end speed number. Only acceptance length is published. Real gains depend on batch size, context length and your kernel stack.
- Narrow hardware story. Blackwell and Linux only per the card. Nothing shows it running on Hopper, AMD or Apple Silicon.
- Target tested as NVFP4. All numbers came from the 4-bit NVFP4 target. The BF16 target may accept differently.
- Licensing stacks. You inherit NVIDIA's agreement and Meta's Muse Model License, including the attribution line and the Model-as-a-Service threshold described in our guide.
- Quality caveats are the base model's. NVIDIA's card repeats that the base model can amplify biases and produce inaccurate text; speculative decoding does not fix that.
- Pre-release tooling. The draft was trained with a pre-release Model Optimizer build, so expect the tooling to change.
What this means for what you build or pay
If you self-host Muse Spark Open on Blackwell GPUs, add the draft to your vLLM launch and benchmark it on your own prompts, especially coding and tool-calling traffic where acceptance is highest. Track tokens per second per user, not acceptance length alone. If you use the model through a cloud endpoint such as SageMaker JumpStart, the provider decides whether a draft is deployed, so there is nothing for you to switch on. If you plan around a Mac or a small cluster, wait for a port: the practical local path today is the quantized route in our Muse Spark Open guide.
Related reading
- Meta Muse Spark Open: 816B open-weight coding model and how to run it
- Meta Muse Glimmer: the 30B open-weight agentic model
- Apple shows 800B-parameter models on four Mac Studios with MLX and RDMA
- DFlash-MLX on Apple Silicon: speedups by model and context
- DeepSeek DSpark and speculative decoding explained
- What is TPS? Tokens per second explained
- Nemotron 3.5 Bolt with a 2-bit drafter
Specs and numbers are from NVIDIA's model card as of October 16, 2026 and may change as the card is updated.