explainx / blog
Ollama 0.31 ships auto-tuned multi-token prediction for Gemma 4 on MLX — 50.2 to 95.0 tok/s on Aider polyglot (M5 Max). Built-in draft model, GPU rollback, and a new MLX batch kernel. On by default for coding agents.
Jun 29, 2026
MacBooks behave like a slow GPU with enormous shared RAM; dedicated cards are fast but VRAM-capped. The right buy depends on whether you wanted a laptop anyway, need privacy at 64k context, or need frontier-speed coding throughput.
Apr 30, 2026
A MIT-licensed desktop app proves local-first vibe coding: Gemma 4 on MLX, build mode with live preview, chat mode with tools, and optional Ollama. Here is the stack, requirements, and what early adopters are asking about.
Jul 1, 2026
Run open weights on your machine, point OpenCode at localhost, and code with no cloud dependency. Full stack — inference server, config file, /connect paths, and when to keep a frontier API in the loop.
Local coding agents live or die on tokens per second. Every file read, tool call, and retry is another decode pass — and on Apple Silicon, 50 tok/s feels like waiting; 95 tok/s feels like a product.
On June 29, 2026, Ollama shipped 0.31 with a headline change: Gemma 4 on MLX is nearly 90% faster on a real coding-agent benchmark — on by default, with identical model output.
| Release | Ollama 0.31+ (macOS) |
| Model | Gemma 4 — first model with integrated MTP (gemma4:12b-mlx) |
| Technique | Multi-token prediction (MTP) — built-in draft model + main verify |
| Benchmark | Aider polyglot (real coding agent tasks) |
| Hardware | M5 Max, Gemma 4 12B nvfp4 |
| Without MTP | 50.2 tok/s |
| With MTP | 95.0 tok/s (~+89%) |
| Config | None — auto-tuned draft length at runtime |
| Output | Unchanged vs non-MTP decode |
| Agent launch | ollama launch claude --model gemma4:12b-mlx |
Autoregressive LLMs generate one token at a time. Multi-token prediction (MTP) adds a small, fast draft model (shipped inside Gemma 4) that proposes the next several tokens. The main model verifies the whole proposal in one pass and keeps tokens it agrees with.
Because the draft model is a small fraction of the main stack, proposals are cheap. When they are correct, you commit multiple tokens for roughly the cost of one forward pass.
Ollama's claim for 0.31:
That last point matters for local model pickers: if you chose Qwen 3.6 + llama.cpp MTP for speed on Mac, Gemma 4 + Ollama 0.31 is now a credible MLX-native alternative — especially if you want multimodal Gemma 4 later.
Ollama calls out code explicitly: closing brackets, repeated identifiers, boilerplate — patterns a draft model guesses well.
That maps directly to loop engineering workloads:
Agent loop: read file → model → tool → model → patch → model → test → model …
Each arrow is decode time. ~90% faster generation does not make the model smarter — it makes Claude Code, Codex, OpenCode, and Droid (via ollama launch) more responsive when pointed at local Gemma 4.
Ollama measured on the Aider polyglot benchmark — a real coding agent running real programming tasks, not a synthetic tok/s microbench. They note explicitly that synthetic benchmarks can be tuned to show almost anything; Aider is their practical reference.
| Setting | Generation speed |
|---|---|
| Gemma 4 12B (nvfp4), M5 Max — without MTP | 50.2 tok/s |
| Gemma 4 12B (nvfp4), M5 Max — with MTP | 95.0 tok/s |
~89% higher throughput on this setup. Your machine, quant, and task mix will differ — treat this as Apple Silicon + Gemma 4 12B + agentic code, not a universal law.
For context:
Ollama 0.31 is specifically an MLX + Gemma 4 + MTP story on Mac.
There is no universal best “draft N tokens then verify.” It depends on model, quant, hardware, and how predictable the text is right now.
Ollama 0.31 tracks acceptance rate and verification time at runtime, picks the draft length that maximizes tokens per second, and reverts to one-token decoding when proposals stop getting accepted. No user-facing knob.
This avoids the classic speculative-decoding foot-gun: speculation enabled but net slower.
Each round:
Rollback is cheap: the engine saves a checkpoint before each proposal and rewinds KV/state to the last accepted token — no full recompute of earlier context.
Drafting, sampling, verification, and post-accept sampling stay on GPU — no CPU ping-pong per token.
Most cost is verification, not drafting. Batches are awkwardly small (2–8 tokens) — between single-token decode kernels and large prefill kernels.
Ollama contributed a kernel to MLX that:
On M5 Max + nvfp4, they report 2×–2.5× speedup on Gemma 4's largest matmuls for that batch shape. Same math, less redundant memory traffic — benefits other MLX models too, not only Gemma 4 in Ollama.
Download from ollama.com (macOS).
If you pulled Gemma 4 before 0.31:
ollama pull gemma4:12b-mlx
ollama launch claude --model gemma4:12b-mlx
Also supported: Codex, Droid, OpenCode, Copilot, and others via ollama launch.
For OpenCode / custom configs, point your provider at Ollama's local API — see run open models in OpenCode.
| Ollama 0.31 (Gemma 4 MLX) | llama.cpp (e.g. Qwen MTP GGUF) | |
|---|---|---|
| Draft source | Built into Gemma 4 weights | --spec-type draft-mtp quants |
| Runtime | MLX via Ollama | llama-server direct |
| Tuning | Auto draft length | Manual flags / model choice |
| Best for | Mac users who want one binary + agents | Max control, router, embeddings |
| Multimodal | Gemma 4 vision/audio path | Text-first today |
Many power users still run llama.cpp directly for -ngl, MTP on Qwen, and OpenCode at localhost:8080. Ollama 0.31 is the low-friction path when you want ollama launch claude and Gemma 4 speed without tuning.
gemma4:12b-mlx pulls may lack MTP-enabled weightsSources: Ollama blog — Faster Gemma 4 on MLX with multi-token prediction (June 29, 2026). Benchmarks: Gemma 4 12B nvfp4, M5 Max, Aider polyglot.