sgl-muse-mega: Meta's Megakernel Pushes Muse Spark Open to 1,109 tok/s

Meta's facebookresearch organization has published sgl-muse-mega, an SGLang plugin that decodes its open-weight Muse Spark Open model with one GPU kernel per decoder layer. The README claims up to 1,109 tokens per second on GSM8K with 8 GB300 GPUs at batch 1, which it puts at 2.1x plain SGLang running the same draft model at the same acceptance length. The code is MIT licensed and marked experimental. This post explains what a megakernel is, what the repo does, how to read the speed claim, and who should care.

The repo is new: one commit, no stars and no forks when we read it on November 5, 2026. We have not run it, because it needs a GB300 cluster. Everything below comes from the repository and the DFlash2 draft model card, plus our earlier coverage.

TL;DR: questions and answers

What is a megakernel, and why use one for decoding?

A normal transformer decode step launches many small GPU kernels per layer: attention projections, attention itself, normalization, expert routing, expert matrix multiplies and so on. At batch 1 each kernel does little work, so launch overhead and idle gaps between kernels take a large share of the time. This is what makes single-stream latency hard to improve even on the fastest chips.

A megakernel fuses many of those steps into one long-running kernel that keeps the GPU busy and overlaps memory loads with compute. We have covered the idea before in Cursor's Mixture-of-Kittens MoE megakernel and in the SGLang plugin system that lets vendors ship their own decode kernels.

What the README says the kernels do

The repo lists these design points. They are the authors' description, and we cannot verify them without the kernel source and a GB300.

• One launch per layer. Each decoder layer runs as 144 blocks of 12 warps, from attention through to the next layer's QKV projection.
• Early weight loading. Weights are loaded early into shared memory or L2 while the previous operation still runs.
• Warp specialization. Some warps load data, some drive tensor cores and some do math, all side by side.
• No fences between GPUs. Readers poll the data itself instead of waiting on a fence.
• Programmatic dependent launch (PDL). The next layer launches early, and each launch warms its own code in L2.
• Speculative decoding in one graph. The DFlash2 draft, the verify and the accept steps run as a single CUDA graph per decode step. A token tree can be enabled with SGLANG_MUSE_MEGA_TREE=1.

The code layout shows the scope: sgl_muse_mega/muse_spark/sm103/ holds device kernels, host launch code and the draft. The sm103 name points at the GB300 GPU generation. A test folder has CPU unit tests, GPU kernel tests and an end-to-end test on a four-GPU node.

How the 1,109 tok/s figure should be read

The README gives a narrow benchmark definition. It is worth reading carefully.

Three points follow.

1. Batch 1 is a latency number. It says how quickly one user's stream can run. It does not say how many users a cluster can serve. A server with many concurrent requests would use different kernels, and the README says those requests fall back to regular SGLang.
2. The baseline is a fair one for speculation, but not the only one. Matching the draft and the acceptance length isolates the kernel gain. It also means the 2.1x is a gain over SGLang's own path, not over, say, TensorRT-LLM.
3. GSM8K is easy math. Acceptance length depends on the task. The DFlash2 model card reports a GSM8K acceptance length of about 5.37 tokens per verification step, against about 3.88 on GPQA Diamond and 4.19 on AIME26. Harder tasks accept fewer draft tokens, so the same kernels would likely yield fewer tokens per second there. That is our inference from the model card's numbers, not a figure from the repo.

"Faster of two runs" also means the figure is a best case, not an average.

The model and draft behind it

Small fast bead running ahead of a larger bead on parallel rails, showing sgl-muse-mega draft and verify speculative decoding

Muse Spark Open is Meta's open-weight 816B-parameter mixture-of-experts model with 42B active parameters. Our guide covers how to run Muse Spark Open, and our post on NVIDIA's DFlash 2 draft for Muse Spark Open explains the speculation side.

The draft used here is RadixArk/Muse-Spark-Open-DFlash2, a roughly 5B-parameter BF16 draft with five Qwen3-style grouped-query attention layers and a 2,048-token sliding window. Per its model card, it proposes tokens in parallel and the target model verifies them. The card's own evaluation used two ARM64 GB300 replicas, each with TP4/EP4 parallelism, and reports results from concurrent suites, not single-request runs. The README for the megakernel instead reports batch 1 on 8 GPUs, so the two sets of numbers measure different things.

How to try it

Only readers with access to GB300 hardware can. The README's quick start is:

pip install -e .
python -m sglang.launch_server --model-path <Muse-Spark-Open-816B-A42B-NVFP4> \
  --tp-size 8 --ep-size 8 --nnodes 2 --node-rank 0 \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path <Muse-Spark-Open-DFlash2> \
  --enable-nvidia-megakernel

This is a shortened version. The full command in the README also sets quantization, the MoE runner backend, attention backend, page size, KV-cache type, context length, memory fraction and reasoning and tool-call parsers. Use the README's exact command, not ours.

Notes from the README:

• On first start the kernels are built with nvcc, which takes seconds, and are cached under the SGLang cache directory.
• Every rank should log NVIDIA megakernel: active when it works.
• The plugin shares SGLang's nvidia_megakernel name with sgl-nvidia-mega, so install only one of them.
• A single-node option uses four GPUs with different settings, including 8 draft tokens instead of 16.
• python benchmark/bench_gsm8k.py --data test.jsonl measures GSM8K through the server.

Why Meta would publish this

Meta has pushed Muse Spark Open as an open-weight model, and an open model is easier to adopt if fast serving is available at release. A kernel plugin that is specific to one model and one chip is a niche asset, but it shows what is possible for the biggest buyers of GB300 racks. It also fits Meta's wider hardware ties: see the NVIDIA and Meta Muse collaboration and the Apple and Meta partnership for local Muse Spark Open inference.

A reasonable read is that Meta wants low-latency decoding for agentic workloads, where a model makes many sequential calls and every call's latency adds up. Faster single streams matter most there.

Who should care, and who should not

Care if you run Muse Spark Open on GB300 for interactive or agentic use, you work on inference systems, or you want a reference for fused MoE decode kernels with speculative decoding.

Do not expect a speedup on consumer GPUs, on H100 or H200, in high-concurrency serving, or for other models. The README does not claim any of those.

For other speculative decoding and kernel work from this season, see EAGLE-4 and DFlash 2 comparisons and Tencent's DSpine drafter.

What this means for what you build or pay

Most teams will not touch this repo directly. The practical signal is about cost and latency at the top end. If single-stream decode on a large open MoE model can run about twice as fast through kernel work alone, then agent loops that chain many sequential model calls get shorter waits without a smaller model. For anyone budgeting GB300 time, it also shows that software choices, not only chip choices, set the latency you get. Until someone reproduces the result, plan with the baseline SGLang number and treat any megakernel gain as upside.

Open questions

• Independent reproduction. No third party has reported results.
• Quality under the token tree. The README claims speed at the same acceptance length. It does not publish accuracy comparisons in the text we read.
• Sampled decoding. The README says sampled decoding is covered, but the headline number is greedy.
• Long context. The draft's configured context is 1,048,576 tokens, but its model card warns that this alignment does not establish acceptance quality across the full context.
• Roadmap. The repo has one commit and a stability warning. We do not know if it will be merged into SGLang core.

Bottom line

sgl-muse-mega is a real, readable code release from a top lab, aimed at a narrow target: batch-1 Muse Spark Open decode on GB300. The 1,109 tok/s and 2.1x numbers are the authors' best-of-two-runs result on a math benchmark. They are credible as an engineering result and not yet confirmed by anyone else. If you have the hardware, test it on your own prompts before relying on the figure.

Related reading

• SGLang plugin system: sgl-nvidia-mega and sgl-amd-mono
• How to run Muse Spark Open 816B
• NVIDIA's DFlash 2 draft for Muse Spark Open
• Cursor's Mixture-of-Kittens MoE megakernel
• EAGLE-4 vs DFlash 2 speculative decoding
• Apple MLX and Muse Spark Open local inference

Repository details are accurate as of November 5, 2026 and may change; the project is marked experimental.