Muse Voice Transcribe: Meta's Real-Time ASR, Diarization, Endpointing in One Model

Meta Superintelligence Labs (MSL) doesn't usually ship infrastructure models with fanfare — Muse Spark and Muse Code get the headlines. On September 1, 2026, that changed. Chief AI Officer Alexandr Wang posted that MSL was "rolling out muse voice transcribe, our first real-time audio perception model - SOTA in streaming speech-to-text," and the benchmark chart backing that claim is not a marginal win. It is a model sitting below the entire prior speed/accuracy frontier.

Mark Zuckerberg posted the same launch from his own account, adding two details worth noting for anyone evaluating this for production use: the model ships with a zero-data-retention tier on the Meta Model API (relevant if you're transcribing anything sensitive), and it supports biasing toward specific names and domain terms — a practical knob for teams transcribing jargon-heavy audio (medical, legal, or company-specific product names) where a general-purpose model would otherwise mis-transcribe uncommon vocabulary.

The headline spec that matters more than the win itself: Muse Voice Transcribe does streaming transcription, speaker diarization, and endpointing in one model call, not three. That collapses a pipeline that most voice-agent and meeting-transcription builders currently assemble from separate vendors.

TL;DR

The architecture: one model, three jobs

Meta's technical writeup describes Muse Voice Transcribe as an autoregressive multimodal model from the Muse Spark family. It processes incoming audio in 80ms chunks (12.5 Hz), converting each chunk into a single soft token. At each step the model chooses between emitting a text token or requesting more audio — a <|next_audio|> token asks for the next chunk, and <|empty_audio|> signals the stream has ended.

That single decoding loop is what lets three traditionally separate systems live in one model:

• Endpointing uses <|speech_onset|> and <|speech_endpoint|> tokens to mark when a speaker starts and stops talking — the "has this person finished their turn" decision that normally sits in a separate voice-activity-detection (VAD) model in front of the ASR stack.
• Diarization uses <|start_of_turn|> to flag a potential speaker switch and <|speaker_{A-Z}|> tags to label who's talking, predicted inline as the stream plays rather than run as an offline clustering pass afterward.
• Transcription is the text tokens themselves, emitted at whatever pace the adaptive delay mechanism decides is safe.

Adaptive delay: the mechanism behind the Pareto jump

The most technically interesting piece is adaptive delay — the system that decides, word by word, how much audio context the model needs before it commits to a transcription. Easy, unambiguous words get transcribed almost immediately. Words that are acoustically or contextually ambiguous get held back until more audio arrives to disambiguate them.

Meta trained this behavior with reinforcement learning that combines a word-error-rate reward with a delay penalty multiplicatively, rather than the more common additive trade-off. That distinction is why Muse Voice Transcribe doesn't just sit on the existing speed/accuracy frontier — it sits below it on both axes simultaneously, which a purely additive objective tends not to produce.

Source: Artificial Analysis, via Meta

The benchmark numbers

Artificial Analysis measured Muse Voice Transcribe against the current streaming speech-to-text field on AA-WER Streaming Index — the percentage of words transcribed incorrectly after the detected end of speech, on final transcription. Lower is better.

For comparison, explainx.ai's own coverage of Gemini 3.5 Transcribe's launch put its streaming WER at 4.0% — Muse Voice Transcribe beats it by nearly a full point in a category where fractions of a percent are contested.

Diarization: the bigger gap

The diarization comparison is where Muse Voice Transcribe's lead widens rather than narrows. Artificial Analysis' average diarization error rate (DER) across the AMI-IHM, AMI-SDM, and VoxConverse benchmarks:

That table is worth sitting with. Muse Voice Transcribe's streaming diarization score beats every competitor's offline score — offline diarization normally has an inherent advantage because it can see the whole conversation before deciding who spoke when. Meta's model gets a lower error rate while running with less information, in real time.

Source: Artificial Analysis, via Meta

Below the Pareto frontier

The scatter plot of AA-WER Streaming Index against time-to-final-transcription is the cleanest single visual of the release. Every prior frontier model — Soniox v5 Real-Time, Cartesia Ink-2, ElevenLabs Scribe v2 Realtime — traces a curve where you trade speed for accuracy or vice versa. Muse Voice Transcribe lands at roughly 0.15 seconds and ~3.0% error, below that entire curve. AssemblyAI U3.5, Grok, Qwen3 ASR, Gemini 3.5 Transcribe Live, and GPT Live Transcribe all sit further right (slower) and higher (worse WER) on the same chart.

Source: Artificial Analysis, via Meta

What "one model, not three" actually changes for builders

Every voice agent, meeting-transcription tool, and live-captioning product built in the last few years has assembled the same pipeline: a VAD model to detect when someone's talking, an ASR model to transcribe it, and a separate diarization model (often run as an offline pass) to figure out who said it. That's three model boundaries, three sets of failure modes, and three places where latency stacks up.

Collapsing that into one model changes the engineering math in a few concrete ways:

• Lower end-to-end latency. Every hop between models costs a network round-trip and a re-buffering step. A single decode loop that emits text, speaker tags, and endpoint markers together removes that entirely — the difference between the ~0.15s Muse Voice Transcribe posts and competitors sitting further right on the Pareto chart.
• Fewer failure points. A three-model pipeline fails when any one component drifts — VAD cutting off a speaker mid-sentence, diarization losing track of a voice after a pause, ASR and diarization outputs falling out of time-alignment with each other. A single model with shared internal state doesn't have a seam for that class of bug to live in.
• Less integration surface. No separate API contracts, no reconciling three vendors' timestamp formats, no debugging which of three services caused a bad transcript. One model, one call, one output schema.
• Native long-session and multi-speaker handling. Meta's claim of hour-plus sessions with 20+ speakers "without post-processing" is a direct jab at the standard workaround — running diarization as a batch job after the call ends because streaming diarization at that speaker count degrades too much to trust live. Muse Voice Transcribe's benchmark numbers suggest that workaround may no longer be necessary.

Where the incumbents still have an edge

None of this makes ElevenLabs, Deepgram, AssemblyAI, or Cartesia irrelevant overnight. Each of those platforms has years of production hardening, broader API surface (voice cloning, text-to-speech pairing, enterprise SLAs), and existing integrations that a benchmark chart doesn't erase. AssemblyAI's offline diarization at 21.1% DER is still a credible number for use cases that can tolerate a post-processing pass and want that vendor's broader tooling. Cartesia's Ink-2 remains the closest competitor on raw streaming WER (3.4% vs 3.1%) and pairs with Cartesia's own TTS stack, which explainx.ai covered at Cartesia Sonic-3.6's benchmark sweep.

What Muse Voice Transcribe changes is the default architecture question. A team starting a new voice-agent build today now has to justify stitching together three vendors instead of reaching for one model that benchmarks better on every axis than the pipeline they'd otherwise assemble.

Availability and where it's already running

Muse Voice Transcribe is live now via the Meta Model API, accessible through developer access at dev.meta.ai. It's already shipping inside two Meta products:

• Meta AI desktop app dictation — explainx.ai covered the Mac app's earlier systemwide dictation feature in Meta AI Mac app: systemwide dictation; Muse Voice Transcribe is the model now behind that experience.
• Muse Code — Meta's coding agent, which exited beta the same day, uses Muse Voice Transcribe for voice input into coding sessions.

Reported pricing sits around $3 per 1,000 audio minutes, roughly $0.18 per hour — in the same range as the streaming tiers from ElevenLabs and AssemblyAI rather than a dramatic undercut. The pitch here is accuracy and architecture simplification, not being the cheapest option on the table.

Honest limitations

• 25 of 70+ languages "extensively validated." The other 45+ are trained but not confirmed to the same bar — benchmark your own target language before shipping, especially for anything outside the six Meta names explicitly (Chinese, French, Hindi, Japanese, Spanish, Vietnamese).
• No independent third-party bake-off yet. Every number in this post traces back to Meta and Artificial Analysis, published the same day as the launch. That's a reputable benchmark source, but it is not the same as a year of production traffic across unrelated teams.
• API-only, no offline/on-device story. This is a hosted model behind the Meta Model API. If your audio can't leave your infrastructure, this release doesn't change your options — see explainx.ai's coverage of local-first alternatives like Meetily's privacy-first local transcription.
• No standalone transcription product. There's no consumer-facing Muse Voice Transcribe app — it's a model you call via API or that runs inside Meta's own dictation surfaces.
• Diarization ceiling untested past 20+ speakers. Meta's own examples demonstrate tracking at least 11 speakers reliably; 20+ is the stated support ceiling, not a number independently stress-tested by outside teams yet.

The takeaway

Muse Voice Transcribe is Meta Superintelligence Labs' first real audio-perception release, and it lands as a genuine state-of-the-art claim rather than marketing rounding: lowest streaming WER measured, lowest diarization error rate measured, and a speed/accuracy position below the field's entire prior Pareto frontier. The architectural point — folding ASR, diarization, and endpointing into one autoregressive model instead of a three-model pipeline — is the part that should change how the next generation of voice agents, meeting tools, and live captioning products get built, more than any single benchmark decimal.

Related on explainx.ai:

• Gemini 3.5 Transcribe: what builders actually get — the closest hosted-API competitor, benchmarked head-to-head above
• Cartesia Sonic-3.6: #1 on both Artificial Analysis TTS boards — the TTS half of a voice-agent stack
• Hugging Face speech-to-speech voice agent guide — where a unified ASR/diarization model fits in a voice-agent pipeline
• Meta AI Mac app: systemwide dictation — the dictation product Muse Voice Transcribe now powers
• Muse Code exits beta — the coding agent that also uses this model for voice input
• Meetily: privacy-first local transcription — the on-device alternative when audio can't leave your infrastructure
• Miso One: 110ms real-time TTS voice model guide — an open-source real-time voice model for comparison
• OpenAI's GPT-Transcribe and GPT-Live-Transcribe — another named competitor in the AA-WER benchmark

Official: Meta AI Research launch post · Artificial Analysis speech-to-text streaming leaderboard · Alexandr Wang's announcement on X

Benchmark figures, pricing, and availability reflect Meta and Artificial Analysis's publicly posted data as of September 1-2, 2026, transcribed from Meta's own launch charts and cross-checked against Meta AI Research's technical writeup. Figures may be revised as third-party evaluation accumulates.