ForeSpark is a new open-source repository in Meta's facebookresearch GitHub organization. It accompanies a paper that attacks a cost most speculative-decoding systems ignore: draft tokens that are generated and then never verified. According to the paper abstract on arXiv, submitted November 10, 2026, ForeSpark improves serving throughput by up to 1.77x over DSpark and 2.49x over autoregressive decoding at a matched per-request generation rate, without changing the output distribution.
The short version: if you run speculative decoding with a parallel drafter under heavy load, a lot of drafting work is thrown away. ForeSpark plans draft length and verification budget together, per request, before drafting starts. The repo was only a day old when we read it, with 2 stars and 1 fork, so treat the numbers as the authors' claims until others reproduce them.
TL;DR: the questions readers will ask
| Question | Answer |
|---|---|
| What is it? | Official code for "ForeSpark: Joint Draft and Verification Budget Scheduling for Parallel Speculative Decoding" |
| Who published it? | The facebookresearch GitHub org. Authors listed: Jaeyong Song, Daniel Wang, Pratyush Patel, Jonny Kong, Preyas Shah, Shobhit Kanaujia |
| Headline result? | Up to 1.77x DSpark throughput, 2.49x autoregressive, at matched per-user speed |
| Under latency limits? | Up to 1.51x DSpark's request rate while meeting the SLOs |
| Does output change? | No. Verification is standard speculative sampling |
| Runs on? | vLLM 0.30.0 with patches; Qwen3-14B, GLM-5.3-Flash, GLM-5.3 |
| License? | Mostly CC-BY-NC 4.0, with Apache 2.0 and MIT parts |
| Production ready? | No release yet; research reproduction code |
What problem does ForeSpark solve?
Speculative decoding speeds up a large model by letting a small drafter guess several upcoming tokens, then having the large target model check them in one pass. We explain the basics in our speculative decoding guide. The term to know here is speculative decoding.
Older drafters were autoregressive: they produced one token after another, so a system could stop drafting early. A parallel drafter proposes a whole block of tokens in a single forward pass. That is fast, but the block length is fixed before the pass runs. The ForeSpark README states the consequence: even goodput-aware systems such as DSpark "verify only the most promising drafts, yet every request still drafts a full block."
The authors measured what that costs. Under load, "drafting takes up to 40% of the step," and most of it is never verified. So a system can be careful about which tokens it verifies and still burn a large slice of every step on tokens nobody checks.
Why goodput matters
DSpark, which we covered in DeepSeek DSpark for V4 Flash and Pro, optimizes goodput: accepted tokens per second, not drafted tokens per second. A rejected draft token still occupies GPU time that another request could have used. DSpark therefore limits how many drafts reach verification. ForeSpark extends that logic one stage earlier, to the drafting step itself.
Continuous batching boxes on a conveyor, an analogy for scheduling draft work across many requests in speculative decoding
How work-conserving speculation works
The paper calls the design work-conserving speculation (WCS). The idea is simple to say: do not spend compute on drafts that will not pay off. Each decoding step runs four stages on a DFlash2 parallel drafter, per the README.
- Pre-draft estimate. A low-rank head reads the target model's context feature and the embedding at each request's anchor. It predicts the chance of acceptance for every candidate draft length. Because the drafter attends bidirectionally, a draft of length d is not a prefix of a longer one, so the head outputs a triangle of K(K+1)/2 logits, one row per length.
- Joint schedule. The scheduler converts those estimates into a draft length D for each request and a total verification budget B. It replaces each request's expected-acceptance curve with its concave envelope, pools the marginal gains of all requests in one global sort, and picks D and B to maximize expected tokens per step time. Latency tables measured on the server set the price of each choice.
- Draft. The drafter runs every request's block at its own length as one packed batch, so drafting cost follows the tokens actually drafted.
- Distribute. A post-draft confidence head re-scores the real drafts, and the verification budget goes to the most promising positions across the whole batch, as in DSpark's adaptive verification.
The host plans totals from the previous step's estimates. The GPU splits them per request with Triton kernels, so the host never waits on the GPU, and every stage replays CUDA graphs. For the GLM targets, the sparse-MLA kernels need each request's lengths on the host, so the host does the per-request split there.
A plain-language picture
Think of a restaurant kitchen that preps a full tray of garnishes for every order, then serves only the best few. ForeSpark asks, before prepping, how many garnishes each order will actually use, and preps only that many. The cooks' time goes to orders that will use it.
What results does the paper report?
All numbers below come from the authors' abstract and README, measured with K = 15 and greedy decoding on the arms in the repository. We could not run them ourselves.
| Metric | Reported result | Compared with |
|---|---|---|
| Throughput at matched per-user speed | up to 1.77x | DSpark |
| Throughput at matched per-user speed | up to 2.49x | Autoregressive decoding |
| Request rate under latency SLOs | up to 1.51x | DSpark |
| Drafting time at the largest batch (ShareGPT-Decode) | 67 to 76% less | DSpark |
| Share of step time spent drafting under load | up to 40% | The problem the paper identifies |
The benchmarks are ShareGPT-Decode (fixed batches, decode only), ShareGPT-E2E (closed loop, with prefill and decode sharing batches) and an open-loop SLO test that scales the request rate. The README also notes that DSpark drafts all 15 tokens per request per step, while ForeSpark drafts "close to what gets verified."
A short fast draft arrow ahead of a longer checking arrow, showing draft and verify stages of ForeSpark speculative decoding
What the numbers do not tell you
- Authors' own setup. The figures use ShareGPT prompts and the repo's own serving configuration with patched vLLM.
- "Up to" is a best case. The 1.77x is the largest gain over the best baseline, marked by arrows in the paper's plots, not an average.
- Matched per-user speed. The gain is at equal interactivity. If you serve one user at a time, the benefit shrinks because there is little batch-level waste to reclaim.
- Load dependent. The whole premise is that drafting hurts most at high serving load. At low load, plain DSpark or simple speculation may be enough.
What is in the repository?
The layout is a research reproduction kit. Thin command-line files (train.py, heads.py, serve.py, data.py, calibrate.py, eval.py, curves.py) call into a forespark/ package. That package holds the drafter and the pre- and post-draft estimators, the scheduler runtime (fore_d/) with Triton kernels and CUDA graphs, serving code, the training launcher and the benchmark harness. Scripts in scripts/ run each stage per target, and plot/ rebuilds the paper's figures.
Targets and hardware
| Qwen3-14B | GLM-5.3-Flash | GLM-5.3 | |
|---|---|---|---|
| Weights / KV cache | BF16 / BF16 | NVFP4 experts / FP8 | NVFP4 experts / FP8 |
| Serving | 8x H100, TP8 | 8x B200, TP8 + EP8 | 8x B200, TP8 + EP8 |
| Drafter (layers x hidden) | 5 x 5,120 | 5 x 4,096 | 6 x 6,144 |
| SLO (TTFT / TPOT) | 1 s / 50 ms | 1 s / 30 ms | 1 s / 40 ms |
This is not hardware most readers own. For context on how a different team pushes an open model on similar Blackwell-class nodes, see our write-up of NVIDIA's DFlash 2 draft for Muse Spark Open, and for engine-level work on the same family of problems, SGL-Muse-Mega.
How to reproduce it
The README gives these commands, one script per stage and target:
./setup.sh # venv with pinned vLLM 0.30.0 + speculators, patches applied
scripts/training/train.sh qwen14 # corpus, then DSpark and ForeSpark drafters
scripts/calibration/calibrate.sh qwen14 # estimator calibration and latency tables
scripts/bench/decode.sh qwen14 # ShareGPT-Decode
scripts/bench/e2e.sh qwen14 # ShareGPT-E2E
scripts/bench/slo.sh qwen14 # open-loop SLO attainment
Training uses Magpie and UltraChat prompts, minus the benchmark's own prompts, with every response regenerated by the target. Each drafter trains online for 3 epochs on hidden states fetched from a vLLM server. A smoke-test mode (DATA_ARGS="--limit 2000" TRAIN_ARGS="--max-steps 20") lets you check the pipeline before spending GPU hours.
The heads-only shortcut
The most practical detail may be the heads-only path. ForeSpark's two estimators can be fitted on an already trained drafter with the drafter and target frozen. On GLM-5.3-Flash, the README reports:
| Trained with the drafter | Heads only | |
|---|---|---|
| Cost | 55 h on 8 GPUs | 1 h on 8 GPUs |
| Confidence head loss / Brier | 0.1773 / 0.0484 | 0.1775 / 0.0486 |
| Fore-D head loss / accepted-token error | 0.511 / 0.688 | 0.522 / 0.729 |
The quality is nearly identical on the confidence head and slightly worse on the Fore-D head, for roughly one fiftieth of the GPU time. If you already run a DFlash-style drafter, this is the cheapest way to test the idea.
How does it compare with DSpark, DFlash and DSpine?
The speculative-decoding field moved quickly this autumn, and each project attacks a different step.
| Project | What it changes | Coverage |
|---|---|---|
| DSpark | Verifies only the most promising drafts (adaptive verification) | DeepSeek DSpark guide |
| DFlash 2 | A block-diffusion style parallel drafter, used as ForeSpark's base | NVIDIA's Muse Spark Open draft |
| DSpine | A parallel drafter that claims longer acceptance than DFlash | Tencent DSpine on Qwen3-8B |
| ForeSpark | Sizes draft length and verification budget jointly per request | This post |
These are not competing in one dimension. DFlash 2 and DSpine improve how good the draft is. DSpark and ForeSpark decide how much of the draft to pay for. A real stack could combine a better drafter with a better scheduler, and ForeSpark itself runs on a DFlash2 drafter.
For readers choosing a serving engine rather than a drafting trick, our vLLM vs SGLang comparison covers the base layer this work patches. For single-machine setups, the same trade-off shows up in llama.cpp speculative decoding on Apple's M3 Ultra.
License, risks and open questions
The license is the first thing to read. The README says the majority of ForeSpark is under CC-BY-NC, with a LICENSE file holding the CC BY-NC 4.0 terms. Files adapted from vLLM and speculators stay Apache 2.0, and layers.py, adapted from Z Lab's DFlash, is MIT. The patches modify vLLM and speculators under Apache 2.0. A non-commercial license generally rules out use in a paid product without separate permission. Ask your legal team before building on it.
Other limits to weigh:
- Pinned stack.
setup.shpins vLLM 0.30.0 and speculators at specific commits and applies patches. Moving a pin means regenerating a patch that no longer applies. - Heavy hardware. The tested setups are 8-GPU H100 or B200 nodes. The paper gives no result for a single consumer GPU.
- No independent replication. The repo appeared on November 11, 2026. We found no third-party benchmark at the time of writing.
- Workload sensitivity. Results come from ShareGPT-style chat. Long-context agent traffic, tool calls or code may shift acceptance patterns.
- Model licenses. The README notes that downloaded models and datasets keep their own licenses and terms.
What this means for what you build or pay
If you rent inference from an API provider, nothing changes today. Gains like this reach you only if a provider adopts the method, and the CC-BY-NC license makes that a question for them. If you operate your own cluster, the scheduling idea is portable even where the code is not: estimate acceptance before drafting, then spend draft and verification compute where it pays.
The broader signal is that serving cost is now fought at the scheduler level. Faster drafters keep arriving, and the next gains come from not wasting what they produce. That matters to anyone watching cost per token for long agentic sessions, a theme in our coverage of Meta's Muse Spark Open.
Practical next steps
- Read the paper on arXiv for the method and the full plots.
- Run the smoke-test settings on the Qwen3-14B target before committing GPU hours.
- If you already have a DFlash-style drafter, try the heads-only fit first.
- Measure at your own load. The benefit is largest under heavy batching, so test at your real concurrency.
- Settle the license question before any commercial use.
Related reading
- What is speculative decoding? An LLM inference guide
- DeepSeek DSpark: speculative decoding for V4 Flash and Pro
- NVIDIA's DFlash 2 draft model for Muse Spark Open
- Tencent DSpine: a parallel drafter vs DFlash
- vLLM vs SGLang: inference server comparison
- llama.cpp Metal speculative decoding on M3 Ultra
- Meta Muse Spark Open 816B: how to run it
- Sources: ForeSpark repository and arXiv 2611.13442
Details reflect the repository README and paper abstract as read on November 12, 2026. The repo is new and may change.