explainx / blog
Mesh LLM v1.0 (July 11, 2026) pools laptops, GPUs, and mini PCs into one OpenAI-compatible API at localhost:9337/v1. Skippy pipeline splits Qwen 235B at 16 tok/s across 2 nodes. iroh QUIC mesh — no central server.
Jul 10, 2026
JustVugg's colibrì streams GLM-5.2 routed experts from a 370 GB int4 container while keeping 9.9 GB dense weights in RAM — 453-point Show HN. explainx.ai maps the physics, honest tok/s, and when disk-streaming beats 256 GB Unsloth setups.
Jul 9, 2026
Docker Desktop founders Jeff and Michael raised $88M for Ollama on July 9, 2026 — 9M+ active builders per @ollama, hybrid cloud scaling, day-one open model support. explainx.ai explains what Ollama is and why it matters now.
Jul 7, 2026
Polymarket and @kimmonismus amplified an r/LocalLLaMA projection: frontier models like Claude Fable reach laptop-runnable open-weight parity in ~24.8 months. explainx.ai unpacks the historical lag math, why you will not run Fable weights locally, and what builders should plan for by mid-2028.
Mesh LLM v1.0 dropped July 11, 2026 — and the pitch from n0's iroh team (Rae McKelvey) is blunt: stop renting racks you don't control. Pool the GPUs already under your desk, expose them as http://localhost:9337/v1, and let the mesh decide whether inference runs locally, routes to a peer, or splits across machines too big for any one box.
No data-center picture. No metered black-box API. Version 1.0 turns that architecture from experiment into something you can install today — ~18 MB node software, 40+ models, and a Hacker News thread already arguing about tokens per second.
| Field | Detail |
|---|---|
| Release | v1.0 — July 11, 2026 |
| API | OpenAI-compatible · http://localhost:9337/v1 |
| Transport | iroh — QUIC, NAT traversal, dial-by-public-key |
| Modes | Local GPU · peer route · Skippy layer pipeline |
| Catalog | 40+ models (0.5B laptop → 235B MoE) |
| Benchmark | Qwen 235B A22B — 16 tok/s across 2 nodes |
| Install size | ~18 MB |
| Roadmap | Mobile app (iroh Swift SDK) · ACP agent standard |
From the iroh blog:
When people picture running a large language model, they picture a data center… You send your prompts off to a black box and hope the price, the model, and the privacy policy all stay the way they were when you signed up.
Teams with GPUs in offices, closets, and home labs lack a way to make those machines act as one inference surface. Cloud APIs scale spend with success; local Ollama stacks cap at single-node VRAM.
Mesh LLM's answer: compose hardware you own into an OpenAI client endpoint — same SDKs, same agents, different economics.
OpenAI client → localhost:9337/v1
│
┌───────────┼───────────┐
▼ ▼ ▼
Run local Route peer Skippy split
(this GPU) (model hot) (layer pipeline)
| Mode | When |
|---|---|
| Local | Model fits this machine's GPU/RAM |
| Peer route | Another node already has weights loaded |
| Skippy split | Model too large for any single node — layer ranges become pipeline stages |
The client never sees Skippy. It still talks to localhost.
Internally "Skippy" partitions by layer range:
Listed perf: Qwen 235B A22B (MoE 235B/22B active) — 16 tok/s across 2 nodes on the models list.
HN contributor @i386 (Skippy engine author): home lab with ~5ms latency + jitter between machines; GLM 5.2 split at ~10 tok/s. Works well at metro WAN latency; global WAN slower. Goal: machines without RDMA/NVLink still serve giants you own — then share with team or public mesh.
Mesh LLM is pluggable: plugins declare capabilities in manifests; runtime routes calls over MCP, HTTP, inference, and mesh events.
Every node boots an iroh endpoint = public key identity. No central coordinator.
| iroh provides | Mesh LLM uses it for |
|---|---|
| NAT hole-punch | Home lab ↔ office peer |
| Relay fallback | Two regional relays when direct path fails |
| Authenticated QUIC | Encrypted peer links (HN: "dial-a-key") |
Three ALPN protocols:
| ALPN | Purpose |
|---|---|
mesh-llm/1 | Gossip, routing, HTTP tunnels, plugins |
mesh-llm-control/1 | Owner config sync, attestation |
skippy-stage/2 | Low-latency activation transport for splits |
Inside mesh-llm/1, the first stream byte demuxes traffic:
| Byte | Stream | Role |
|---|---|---|
0x01 | GOSSIP | Peer announcements (models, GPU, RTT) |
0x04 | TUNNEL_HTTP | Inference proxied to peer |
0x05 | ROUTE_REQUEST | "Which models do you host?" |
0x06 | PEER_DOWN | Dead-peer signal |
0x07 | PEER_LEAVING | Graceful shutdown |
0x08 | PLUGIN_CHANNEL | Plugin RPC |
0x0e | DIRECT_PATH_REQUEST | NAT traversal address share |
explainx.ai read: iroh makes "peer" and "localhost" the same primitive with different endpoint IDs — networking stops being a custom ops project.
Mesh LLM builds its own gossip on top to control admission, version compatibility, and trust for private meshes.
SwellJoe (HN): consumer networks are slow vs local RAM — "Are we talking 1 token per second?"
Counter-evidence:
| Data point | Source |
|---|---|
| 16 tok/s · Qwen 235B · 2 nodes | Mesh LLM models list |
| ~10 tok/s · GLM 5.2 split | i386 · 5ms lab latency |
| Orders of magnitude faster than llama RPC | i386 on staged Skippy vs naive RPC |
woadwarrior01 ballpark: transfer roughly 2 × hidden_size × num_shards bytes per token during decode (divide by chunk size on prefill).
Reality check: 16 tok/s is usable for agents, borderline for chat UX — comparable to why teams still pay for GPT-5.6 Sol cloud tokens. Mesh LLM wins on privacy, lock-in, and marginal cost at scale — not raw interactive speed vs a single H100.
For single-node baselines: MacBook vs dedicated GPU local LLM guide.
Per the July 11 announcement:
http://localhost:9337/v1Coming: mobile app on iroh Swift SDK; ACP (agent standard) so more clients join without proprietary lock-in.
Philosophy (blog): "more peer to peer, fewer closed servers, and no lock-in."
| Approach | Multi-node split | OpenAI API | P2P encrypted | Central server |
|---|---|---|---|---|
| Mesh LLM v1.0 | ✅ Skippy | ✅ :9337/v1 | ✅ iroh QUIC | ❌ |
| Ollama | ❌ | ✅ (single host) | N/A | ❌ |
| vLLM | ❌ (typical) | ✅ | N/A | self-hosted |
| OpenAI / Anthropic API | N/A | ✅ | TLS to vendor | ✅ vendor |
| cocompute.ai | 🚧 (HN: round-robin pool today) | varies | varies | pool coordinator |
HN noted cocompute for trusted-pool delegation; model splitting across devices is what Mesh LLM v1.0 ships now.
| Profile | Fit |
|---|---|
| Home lab / office GPU fleet | Run 235B-class without one $40k box |
| Privacy-sensitive teams | Data never leaves your mesh |
| Agent builders | Same OpenAI SDK as Kokoro TTS stack patterns |
| Cost escapees | Fable credits July 13 → owned-hardware inference |
| Interactive chat only | Cloud or single-GPU local still faster |
Pair with Fable 5 local hardware projections for the macro trend: frontier models eventually fit owned silicon — Mesh LLM accelerates today's gap via networked sharding.
Not a replacement for MCP security hardening when agents call tools on mesh-served models.
Official: Mesh LLM blog — iroh.computer · iroh docs · HN discussion · Mesh LLM website (linked from iroh blog)
Throughput depends on GPU tier, quantization, network latency, and split tuning — treat 16 tok/s as a published data point, not a SLA. Verify hardware requirements on the Mesh LLM models list before production workloads.