Fermion Research shipped Neutrino-1 8B on July 27, 2026: one coded ternary-family container (~3.88 GB on disk, ~2.56 GB download) meant to serve a datacenter GPU, a MacBook, and a desktop CPU without three different conversions.
It sits in a hot niche — PrismML Bonsai, BitNet-style 1.58-bit work, Hy3 extreme GGUF — and landed on HN (~29 points) next to essays about owning endpoints and owning specialists. Density is the third ownership story: own a model small enough that the path is yours.
TL;DR — What People Are Asking
| Question | Answer |
|---|---|
| What? | Neutrino-1 8B ternary-family open weights |
| Base | Qwen3-8B · Apache 2.0 |
| Size | ~3.88 GB container · ~2.56 GB download |
| Params | 8.19B (6.95B coded linears + int8 embeds) |
| MMLU | 72.1 (lab, thinking off) |
| Speed | H100 ~396 tok/s · drafted up to ~763 |
| Laptop | M5 MLX ~33.7 tok/s · CPU ~24.9 |
| Install | pip install fermion-research → fermion chat |
| Skepticism | “Native ternary?” · AI-ish docs · vs PrismML Bonsai |
Architecture Snapshot (From the Model Card)
| Spec | Value |
|---|---|
| Type | Dense decoder-only transformer |
| Layers | 36 |
| Hidden | 4096 · FFN gated SwiGLU 12288 |
| Attention | GQA 32Q / 8KV · head 128 |
| Context | 40,960 tokens |
| Vocab | 151,936 · untied embeds (int8) |
| KV cache | ~144 KiB/token fp16 · ~0.60 GB @ 4k |
| Coded lane | 252 ternary-family linears · ~67% of file |
| Zeros in codes | ~62.6% overall (lab) |
Only transformer linears use the proprietary coded format; embeddings stay int8; norms stay float32. Serving economics pitch: smaller working set → higher single-stream decode on the same memory bus, and room for KV on 8 GB GPUs / 16 GB laptops.
Three Doors, One Artifact
- pip / fermion engine — native macOS arm64 & Linux x86-64 binaries + torch reference.
- GGUF + Fermion llama.cpp fork — CUDA offload (~30.7 tok/s on L4 in their table).
- MLX — Metal kernels, memory-mapped container (~33.7 tok/s on M5).
Plus Neutrino-1 0.6B as a speculative draft (and a fast model alone). Lab claims drafted output matches plain greedy (27,648 tokens, zero divergences in their certified run). Acceptance — and thus tok/s — varies by prompt class (counting ≫ code).
Eval Card (Lab Harness, Thinking Off)
| Bench | Score |
|---|---|
| MMLU 5-shot | 72.1 |
| MMLU-Redux | 67.8 |
| IFEval prompt-strict | 77.2 |
| IFEval instruction-strict | 80.2 |
| BFCL v3 macro | 68.9 |
| GSM8K flexible | 53.4 |
| GSM8K stated format | 51.73 |
Treat as release-battery numbers, not agent-loop reality. Tool calling and multi-turn harnesses are where extreme compression usually bleeds — the same lesson from Bonsai HN notes.
HN’s Reality Check
Useful skepticism from the Neutrino thread:
- BitNet / HF 1.58-bit fine-tune recipes have been public for years — “proprietary native ternary” may be incremental.
- PrismML Ternary-Bonsai-8B may outperform on several charted metrics; independent tests of PrismML already saw drop-offs off cherry-picks.
- Release prose looked AI-generated to some readers; that does not invalidate weights, but it raises the bar for trusting unpublished training claims.
explainx.ai rule: download → run your eval → decide. Do not migrate production agents on a model card alone.
Where Neutrino Fits in a Stack
| Goal | Prefer |
|---|---|
| Max quality coding agents | Frontier / large open (Kimi, GLM) hosted |
| Cheap local chat / RAG assist | Neutrino / Bonsai / Qwen QAT |
| Phone extreme density | PrismML 1-bit paths (with vision caveats) |
| Spec decode experiments | Neutrino 8B + 0.6B pair |
| Known llama.cpp toolchain | Mature GGUF quants until fork stabilizes |
Ownership arc for the week: feel the endpoint → own the specialist → fit the weights in your bag.
Quick Start
pip install fermion-research
fermion chat
# or
fermion serve # http://127.0.0.1:8000/v1 OpenAI-compatible
Draft pairing and GGUF/MLX paths are documented on Fermion’s model and engine pages. Cite their BibTeX if you publish results.
Ternary / 1.58-bit Context (Why This Niche Exists)
BitNet-style research showed networks can live near ternary weights (minus one, zero, plus one) if trained for it. Post-training extreme quantization of fp16 models is harder; gradual fine-tune recipes (Hugging Face’s 1.58-bit notes) and labs claiming native coded formats both chase the same prize: memory-bound decode.
If matrix multiply is limited by bytes moved per token, an ~4 GB working set beats a 16 GB fp16 twin on the same bus. That is why Neutrino brags about 8 GB GPUs and 16 GB MacBooks — not because 8B ternary equals Opus, but because good-enough local becomes affordable.
PrismML’s Bonsai line (including 8B ternary variants discussed on HN) is the closest competitor narrative. Expect weeks of community bake-offs on:
- tool calling / BFCL-style multi-turn,
- coding agent harnesses (OpenCode, Continue, Aider),
- long-context degradation,
- and whether custom CUDA/MLX forks stay maintained.
Until then, keep a known llama.cpp Q4/Q5 Qwen or Gemma as your control.
Eval Protocol You Should Run (Not Theirs)
| Test | Pass bar |
|---|---|
| 20 real coding diffs from your repo | ≥70% shippable with light edit |
| 10 tool-using agent tasks | No silent tool hallucination |
| 4k and 16k context summarization | Faithful, no loop (looping fix guide) |
| Spec-decode on / off | Token-identical outputs |
| Thermal / battery on laptop | Usable for 30+ min session |
If Neutrino fails tool tasks but wins chat latency, use it as a draft / classify tier behind a stronger planner — the same split as Fermisense’s frontier-then-specialist story.
Byte Budget Mentality
Fermion’s own breakdown is a good teaching tool: ~67% of the file is coded linears, ~32% is int8 vocabulary, norms are noise. Extreme compression only pays if the hot path (matmul weights) shrinks. Vocab stays fat because rows are gathered, not GEMMed at full width every step.
That is why “8B ternary” still costs gigabytes — and why a 0.6B draft at 328 MB is a cheap bolt-on. Speculative decoding is not free magic; it is acceptance physics. Counting prompts draft well; code drafts poorly. Measure per class or you will advertise 763 tok/s and ship 426.
Relationship to “Open Feels Good”
Saltz’s Modal endpoint is cloud ownership of the path. Neutrino is ownership of the artifact small enough for a backpack. Fermisense is ownership of the weights after your data. Different layers; same political economy: stop treating intelligence as a single rented oracle. Our open-source AI position is the classroom version of that stack.
Install Surfaces Checklist
| Surface | Best for | Watch-outs |
|---|---|---|
fermion chat / serve | Fastest “does it run?” | Binary trust; first download pulls container |
| GGUF + fork | CUDA boxes already on llama.cpp muscle memory | Fork drift vs upstream |
| MLX pack | Apple Silicon laptops | Python env + Metal quirks |
| HF weights alone | Research / custom loaders | Need their engine for coded format |
Commercial use is Apache 2.0 per their card (derivative of Qwen3-8B). Still read the license before bundling in a product.
Bottom line for builders: Neutrino is a density bet, not a frontier replacement. If the artifact runs cleanly on your Mac and passes a thin agent eval, it earns a slot next to Bonsai and Qwen QAT. If the prose around it feels overfit to SEO, ignore the prose — keep the weights under A/B test for a week before you delete your known-good GGUF baseline from disk.
Honest Limitations
- Lab benches ≠ your agent harness.
- “Native ternary” is currently soft-claimed.
- Fork / custom format = tooling lag (LM Studio, stock llama.cpp).
- GSM8K mid-50s is not “frontier math.”
- Small HN sample — wait for third-party replications.
Related on explainx.ai
- PrismML Bonsai 27B / ternary phone models
- What is llama.cpp?
- MacBook vs dedicated GPU for local LLMs
- Tencent Hy3 1-bit / 4-bit GGUF
- Using an open model feels surprisingly good
- Intelligence ownership — Fermisense GRPO
- Top open-weight models on a laptop
- How to run open models in OpenCode
Primary sources: Fermion Neutrino-1 8B model page · Neutrino Engine / research · Hugging Face Fermion Research repos · HN discussion · BitNet b1.58
Specs and throughput figures are Fermion’s July 27, 2026 release card. Verify on your hardware and harness. Follow @explainx_ai for independent follow-ups.