Ternlight: 7 MB Embedding Model That Runs in the Browser (WASM SIMD Guide)
@ternlight/base ships a 7 MB sentence encoder in Rust→WASM — ~5 ms/embed, no API. Distilled from MiniLM with ternary quantization. npm, React docs demo, vs gte-small.
Semantic search usually means: embed documents on a server, embed the query via an API, pay per token, pray your RAG pipeline stays up. Ternlight flips the default — a 7 MB sentence encoder that runs in the browser via Rust → WASM SIMD, returning 384-dimensional vectors in ~5 ms with zero network after load.
The live demo indexes 2,000 React documentation chunks entirely on-device. Type a question; cosine similarity ranks matches even when wording diverges ("reset my password" ↔ "I forgot my password" → ~0.88). For anyone building FAQ bots, support search, or privacy-first apps, Ternlight is the kind of small model that makes embeddings feel like infrastructure, not a SaaS line item.
Embeddings encode meaning as dense vectors. Classic workflow:
text
User query → API embed → vector DB ANN search → top-k chunks → LLM
Pain points:
Latency — network + cold starts on every keystroke in search-as-you-type
Cost — embedding APIs charge per token at scale
Privacy — support tickets and internal docs leave the device
Offline — mobile and air-gapped environments
Ternlight targets small-to-medium corpora (hundreds to low thousands of chunks) where client-side embed + brute-force cosine is fast enough after one-time indexing.
Why not transformers.js or ONNX in browser? Ternlight optimizes for one job — short sentence embeddings — at single-digit megabytes. General runtimes ship broader ops at higher weight cost.
Quick start — three lines
bash
npm install @ternlight/base
javascript
import { embed, similar } from'@ternlight/base';
const recipes = [
'quick pasta with garlic and olive oil',
'slow cooker beef stew for winter',
'15-minute stir fry with rice noodles',
];
similar('easy weeknight dinner ideas', recipes, { topK: 3 });
// → ranked matches · ~5 ms · zero network
@ternlight/mini — smaller wire size, ~2.5 ms/embed — trade accuracy for speed in tight UI loops.
HN insight: Precompute embeddings server-side once, ship vectors JSON to the client, run query embed + cosine locally — best of both worlds for larger corpora.
wazzup.im integration (HN comment): offline search at app.wazzup.im/search — first visit downloads model; later runs from cache.
gte-small scores higher on MTEB (~61 vs MiniLM ~56)
Ternlight holds 0.84 Spearman fidelity to MiniLM teacher
Head-to-head STS-B/MTEB — on roadmap
gte-small distillation — also on roadmap
Dimension
Ternlight @ternlight/base
OpenAI text-embedding-3-small
Self-hosted gte-small
Size
7 MB in bundle
N/A (API)
~130MB+ model files
Network
None after load
Required
Self-host
Cost
Free (MIT)
Per-token
GPU/CPU ops
Latency
~5 ms local
50–300ms+ RTT
Depends on hardware
Quality
Good; not SOTA
Strong
Stronger on benchmarks
explainx.ai read: Ternlight is a wedge product — prove useful embeddings without ops. Pair with a small local LLM workflow for full on-device Q&A.
WASM, SIMD, and the W3C LLM API question
Ternlight rides WebAssembly SIMD — near-native CPU math in Chrome, Firefox, Safari (modern versions). HN tangents asked for a W3C LLM API (Language.complete(...)). Embeddings are simpler: deterministic forward pass, no sampling, no streaming tokens — ideal first citizen for standardized in-browser inference.
Browser matrix (July 2026): SIMD WASM is widely available on desktop; iOS Safari supports WASM but test memory limits on older iPads when loading 7 MB weights + 2k×384 float32 vectors (~3 MB vectors alone).
Practical 2026 pattern:
text
Static site / SPA
├── ternlight.wasm + weights (7 MB)
├── precomputed embeddings.json (server build step)
└── query: embed locally → rank → optional LLM API for answer synthesis
Separation of concerns: cheap local retrieval, expensive cloud generation only on the top-k.
Training pipeline — why open weights matter
Unlike black-box API embedders, Ternlight ships:
MIT license on code + weights
Training pipeline in repo — reproduce or fine-tune on domain data
Distillation recipe documented for ML engineers auditing compression
For regulated industries, on-device + auditable weights beats "trust our API" — aligns with PII tooling patterns (keep text local, embed local, redact before any cloud LLM).
Limitations and roadmap (honest)
From GitHub issues and HN:
Benchmarks — public STS-B/MTEB table still pending
gte-small teacher — planned upgrade path
Initial index time — ~30s for 2k chunks surprises users; document pre-indexing
Fan noise — demo maxes CPU; HN joked about laptop fans — use Web Workers in production
Not generative — needs separate LLM for answer composition
When to choose Ternlight in your stack
Choose Ternlight if:
Corpus under ~10k chunks and brute-force cosine is OK
Privacy/offline is non-negotiable
You want search-as-you-type without embedding API bills
You are prototyping before committing to Zvec or Pinecone
Skip Ternlight if:
You need multilingual SOTA on long-tail languages (verify first — not marketed for 100+ langs)
Your corpus is massive — invest in ANN + server embedders
Sub-1% retrieval recall matters for legal/medical — benchmark against cloud baselines first
Benchmarks and evaluation — what to measure before shipping
Before swapping OpenAI embed calls for Ternlight in production, run a domain eval (same method as embeddings guide):
Sample 100–200 real user queries from support logs or search analytics
Label 3–5 relevant docs per query from your corpus
Embed corpus once (server or client — store vectors in IndexedDB)
Measure Recall@5 and NDCG@10 vs your OpenAI baseline
Expectation setting: Ternlight targets ~0.84 Spearman fidelity to MiniLM teacher — not MTEB leaderboard wins. For internal docs in English, that is often good enough for search-as-you-type; for legal retrieval, run side-by-side.
Pre-indexing pattern (HN-recommended):
text
CI job: embed all docs with @ternlight/base (Node)
→ upload embeddings.json + manifest to CDN
SPA: fetch vectors on first visit (cache)
→ embed query in WASM on each keystroke (debounced)
→ rank top-k locally
This splits one-time O(n) indexing from interactive O(n) query — acceptable until ~10k chunks, then shard or move ANN server-side.
Web Workers and production hygiene
The demo maxes a CPU core — users on HN noted laptop fans spinning on page open. Production checklist:
Run embed() inside a Web Worker — keep UI thread free
Debounce keystrokes (150–300 ms) before re-ranking
Cache model in CacheStorage or IndexedDB after first load
Show indexing progress — 30s for 2k chunks surprises users who expect instant search
Fall back to keyword search while WASM loads
What people are asking (from HN)
Question
Answer
vs gte-small?
gte-small wins benchmarks; head-to-head not published yet; gte distillation planned
Can I precompute server-side?
Yes — ship vectors to browser, query locally
W3C LLM API?
Embeddings are easier to standardize than generative LLMs — deterministic forward pass
Math/coding search?
wazzup.im added offline search — model small, not SOTA for code
Why not transformers.js?
Ternlight optimizes one task at 7 MB — general runtimes are heavier
Integration with RAG and agent stacks
Ternlight slots into the retrieval half of a RAG loop without touching generation:
This mirrors explainx.ai's local AI workflow — cheap retrieval local, expensive reasoning remote. For Langflow prototypes, swap the embed node with a small custom component calling @ternlight/base before you commit to Pinecone bills.
Agent memory angle: episodic memory systems often embed conversation summaries. A 7 MB embedder in the browser could rank which past sessions to inject into context — same primitive as vector search fundamentals, different packaging.
Mobile PWA note: 7 MB is viable on mid-tier phones after Wi‑Fi first load; test thermal throttling on sustained indexing — same fan issue as laptop demo.
Licensing and forkability
MIT license covers code + weights + training scripts — rare for embedding products. Teams can:
Fine-tune on domain corpora (support macros, legal clauses, medical intake forms)
Ship white-label search widgets without API rev-share
Audit quantization impact on compliance-sensitive vocab
Contrast with OpenAI embed APIs where weights are inaccessible — Ternlight is an inspectable primitive for security reviews.
Roadmap watch: creator noted gte-small distillation and public STS-B/MTEB tables on GitHub — when those land, re-run your domain eval before assuming MiniLM-era fidelity caps.
FAQ — quick answers for engineering leads
Can we drop Pinecone for internal docs? For under ~5k chunks with privacy requirements, yes — pair Ternlight with IndexedDB vectors. Keep Pinecone when you need hybrid BM25, multi-tenant isolation, or billion-scale ANN.
Does this replace OpenAI embed in production RAG? Only after your Recall@5 eval passes on real queries. Many teams run Ternlight in the browser for search UI while keeping server embedders for batch ingest until pipelines are unified.
Security review blocker? MIT weights + WASM sandbox is easier to approve than sending all doc text to a third-party embed API — document the WASM SIMD CSP requirements for your InfoSec team.
Model sizes, latency, and fidelity figures reflect the July 2026 release. Run your own STS-B eval on domain data before replacing production embedders.
