EAGLE-4: One Flexible Speculative Drafter, 3.59x Faster Qwen3.8-27B on B300

Speculative decoding is the most practical way to make a large language model generate faster without changing its answers, and the EAGLE line of methods has been a reference point for it since 2024. Now the EAGLE team has released EAGLE-4, which it describes as "one flexible drafter for all batch sizes and hardware." On Qwen3.8-27B with four NVIDIA B300 GPUs, its model card reports 863.8 tokens per second and a 3.59x speedup over vanilla decoding at temperature 0.

This guide explains what EAGLE-4 claims, how the comparison tables should be read, and what to check before adopting it. Source: the EAGLE-4 draft model card for Qwen3.8-27B and the EAGLE-4 GitHub repository.

TL;DR: EAGLE-4 at a glance

What is speculative decoding, in one paragraph?

A large model produces one token per forward pass, and each pass is limited by memory bandwidth, not arithmetic. Speculative decoding adds a small, fast draft model that proposes several candidate tokens; the large target model then checks them all in one pass and keeps the ones it would have produced anyway. When the guesses are good, you get several tokens for the price of one pass. Because the target verifies every token, the output distribution is unchanged: you get the same quality, faster. The model card repeats that EAGLE-4 maintains "the target model's output distribution through speculative verification." For a primer on speed metrics, see our explainer on tokens per second.

What is different in EAGLE-4?

Earlier drafters were typically tuned for a particular regime: great at batch size one, less so when a server batches many requests, where verification is already compute-bound and wasted drafts cost more. EAGLE-4's pitch is a single drafter that "dynamically adapts its speculative decoding strategy to different batch sizes, hardware platforms, and workloads." The card does not give architectural detail, pointing instead to a paper PDF in the repository. We did not review the paper, so we cannot describe the adaptation mechanism beyond that sentence.

The benchmark table, read carefully

The model card's tables use the qualitative split of NVIDIA's SPEED-Bench, concurrency 1, four B300 GPUs with tensor parallelism 4, maximum output length 4,096 tokens, and the same inference engine for all methods with MegaKernel disabled.

Temperature 0:

Temperature 1:

What the numbers do and do not show:

• The gap narrows with sampling. At T=0 EAGLE-4 leads DFlash 2 by about 28%; at T=1 by about 9%. Sampling at temperature 1 makes drafts harder to accept, which hurts every method, and chat workloads often use nonzero temperature.
• Concurrency one only. The tables say nothing about high-concurrency serving, which is exactly where the "all batch sizes" claim would matter. Ask for those numbers or measure them yourself.
• Same engine, MegaKernel disabled. That keeps methods comparable but understates what a fully optimized stack might reach. The GitHub README reports per-engine speedup ratios (for example 3.42x for SGLang with one request) that are separate runs from the card's table.
• Vendor-run. The EAGLE team ran the comparison, and competitor drafters can be tuned differently. DFlash 2 is covered in our post on NVIDIA's DFlash 2 draft model for Muse Spark Open.

A short history of the EAGLE family

The model card's citation list is a compact timeline. The first EAGLE paper, "Speculative Sampling Requires Rethinking Feature Uncertainty," appeared at ICML 2024 and drafted at the feature level instead of the token level. EAGLE-2 ("Faster Inference of Language Models with Dynamic Draft Trees") followed at EMNLP 2024, adding dynamic draft trees that adjust to how confident the drafter is. EAGLE-3 ("Scaling up Inference Acceleration of Large Language Models via Training-Time Test") was published at NeurIPS 2025. EAGLE-4 is listed as a preprint from the "Eagle AI Team," with the earlier papers credited to Yuhui Li, Fangyun Wei, Chao Zhang and Hongyang Zhang.

Each generation has been picked up by serving engines, which is why plugin support for vLLM, SGLang and TokenSpeed is part of the pitch: a drafter is only useful if it drops into the stack you already run. The competing approaches in the table, DFlash 2, DSpark and MTP (multi-token prediction heads built into the target model), come from other teams and have their own strengths. MTP, for example, needs no separate draft model at all but only exists for models trained with it.

Reading the speedup numbers like an engineer

A 3.59x speedup does not mean 3.59 accepted tokens per step. Throughput gain is the number of tokens produced per target pass, divided by the relative cost of the draft work and verification overhead. If a drafter proposes eight tokens and the target accepts an average of four, you get roughly four tokens per target pass, minus the time spent drafting. Hitting 3.59x at concurrency one on a 27B model across four GPUs suggests high acceptance and low drafting overhead on that benchmark.

Two practical consequences follow:

1. Acceptance depends on the text. Code and structured output are easier to predict than creative writing, and benchmark sets like the one used here are a mixture. Your acceptance rate on your traffic will differ.
2. Overhead grows with batch size. At high concurrency the target is already busy, so rejected drafts cost real throughput. That is the regime where the card's "all batch sizes" claim needs independent confirmation.

Decision guide

A clock on a schedule, illustrating the per-token decoding time that EAGLE-4 speculative decoding tries to cut

How to try it

1. Read the license in the EAGLE-4 repository and the Qwen3.8-27B license before deploying.
2. Install the EAGLE-4 code, which includes the eagle-harness package for SGLang, vLLM and TokenSpeed according to the README.
3. Download the matching draft model: EAGLE-Corp/Qwen3.8-27B-EAGLE4. The card stresses that it is not a standalone text-generation model; it needs the target model and the EAGLE-4 code.
4. Benchmark on your own prompts, at your own concurrency, with your own sampling settings, comparing against your current speculative method (such as MTP or a DFlash drafter) and vanilla decoding.
5. Check acceptance rate and tail latency, not only average throughput.

For other approaches to the same problem, see our coverage of Tencent's DSpine drafter, speculative decoding in llama.cpp on Metal, and the TokenSpeed engine.

Why this matters for agent workloads

Agent loops are latency-bound: each step waits on a generation, then a tool call, then another generation. Cutting decode time by two to three times shortens whole runs, which is why inference teams treat speculative decoding as a core feature, not an optional extra. It is also why the engine ecosystem is crowded with competing drafters and kernels. If you run coding agents on self-hosted models, speed at low concurrency is exactly what the EAGLE-4 benchmark measures, which makes it relevant, with the caveat that the vendor ran it. Measure end-to-end task time on your own agent traces rather than raw tokens per second.

Questions to ask before switching

• Is a custom license acceptable? The card calls it "eagle-4" and links the repository license. It is not Apache or MIT, so legal review applies.
• Will it help at your batch size? If you serve many concurrent users, test at that concurrency.
• Does it support your model? Public drafts so far cover Qwen3.8-27B and a Nemotron 3.5 Super VL model; other targets need a trained drafter.
• Is your bottleneck decoding at all? For long prompts or heavy tool use, prefill, caching and network time can dominate.

Related reading

• DFlash 2 draft model for Muse Spark Open
• Tencent DSpine speculative decoding drafter
• TokenSpeed inference engine
• llama.cpp Metal speculative decoding
• What is TPS?
• Meta Dist-MoE kernels
• Official: EAGLE-4 on GitHub and EAGLE-Corp on Hugging Face

Details are accurate as of October 22, 2026 and come from the model card; benchmarks were run by the EAGLE team.