The open-weight leaderboard is full of models that technically “run locally” if your local machine is a multi-GPU server. A useful laptop list needs a harsher test: after quantization, operating-system overhead, context cache, and usable generation speed, would a normal person keep the model installed?
This ranked guide covers 8GB, 16GB, 24GB, and 32GB machines. It extends our editorial case for open-weight AI into something readers can use this week. It does not call every downloadable model “open source”; licenses differ, and open weights may still restrict use.
TL;DR: picks by memory tier
Question
Direct answer
8GB machine?
Phi-4-mini, Gemma 3 4B, or Qwen 3 4B at 4-bit
16GB machine?
Qwen 3.5 9B, Gemma 12B-class quants, or low-bit GPT-OSS-20B
24GB machine?
GPT-OSS-20B Q4 with context room; 14B-class reasoning models
32GB unified memory?
27B–35B low-bit models become practical, though not always fast
Best runtime for beginners?
Ollama or LM Studio; llama.cpp for more control
Biggest buying mistake?
Fitting weights exactly and leaving no memory for context or the OS
The memory math before the ranking
A dense model's raw weight memory is approximately:
An 8B model at 4-bit is about 4GB before metadata and runtime overhead. A 20B model at 4-bit is about 10GB. Real use adds the KV cache, temporary buffers, model metadata, runtime, and operating system. Longer context expands cache memory; multimodal models may load additional components.
Use this conservative rule:
Installed RAM / unified memory
Target quantized weight file
8GB
2–4GB
16GB
5–10GB
24GB
10–16GB
32GB
14–22GB
Dedicated VRAM and system RAM are not equivalent. GPU-resident models are faster; partial CPU/system-memory offload can work but may sharply reduce tokens per second. Apple unified memory gives the GPU access to a larger common pool, though bandwidth still limits speed.
The top 10 laptop open-weight models
1. GPT-OSS-20B — best serious 16–24GB generalist
OpenAI's 20B open-weight model is the 2026 default recommendation when you want the strongest general capability that still belongs in the laptop conversation. A 4-bit file needs roughly 10–13GB depending on format, making 16GB possible but tight and 24GB comfortable. On 16GB, reduce context and close memory-heavy apps.
Best for: reasoning, everyday writing, coding, and tool experiments. Avoid when: maximum speed matters more than quality or you need a huge active context on 16GB.
2. Gemma 4 12B — best multimodal quality/size balance
Google's Gemma family has been consistently strong for single-device use, and the 12B class remains the practical sweet spot for text plus multimodal work. A Q4 quant generally fits a 16GB unified-memory machine with reasonable headroom if context is controlled.
Best for: document/image understanding, writing, and general assistance. Watch: license terms, runtime support for the exact multimodal build, and image encoder overhead.
3. Qwen 3.5 9B — best responsive 16GB all-rounder
Qwen's 9B class leaves more room for context than 12B or 20B alternatives and performs well across code, multilingual work, structured output, and tools. It is the safer daily driver when GPT-OSS fits only by starving the rest of the system.
Best for: local agent tools, multilingual chat, code, and RAG. Watch: thinking modes can increase output length and latency.
4. Ministral 3 8B — best efficient reasoning alternative
Mistral's small-model line targets edge and local deployment with capable instruction and reasoning variants. At 8B, a good 4-bit quant can run on 8GB with severe constraints and feels much healthier on 16GB.
Best for: structured business tasks, summaries, light reasoning, and European deployment preferences. Watch: pick the instruction/reasoning variant that matches your workload rather than assuming one tag does everything.
5. Qwen 3 8B — best mature ecosystem fallback
The earlier Qwen 3 8B remains valuable because runtimes, quantizations, prompts, and community knowledge are mature. Newer does not always mean easier. For a stable local deployment, a widely tested model can beat a preview with marginally better benchmarks.
Best for: coding, tools, multilingual work, and learning local serving. Watch: benchmark gains from long reasoning can trade away interactive speed.
6. Phi-4-mini — best for 8GB and fast utility
At roughly 3.8B parameters, Microsoft's Phi-4-mini class is a realistic model for a base laptop. A 4-bit quant can land around 2.5–3GB, leaving room for a moderate context and the operating system.
Best for: extraction, classification, rewriting, simple code help, and local privacy. Avoid when: the task needs broad factual recall, complex planning, or long-horizon agents.
7. Gemma 3 4B — best small multimodal learner model
Gemma 3 4B remains an accessible way to experiment with multimodal local AI on modest hardware. It is not the strongest model on this list; it is one of the easiest to run with enough headroom to learn how context and quantization affect behavior.
Best for: basic image/document tasks, summaries, and education. Watch: small-model confidence can exceed its knowledge—require evidence.
8. MiMo-7B — best compact new entrant
Xiaomi's MiMo-7B family is an open-weight 2026 entrant built for strong capability at small scale. Seven billion parameters puts it in the comfortable 16GB tier and the constrained 8GB tier at 4-bit.
Best for: experimentation, reasoning variants, and comparing Chinese open-weight ecosystems. Watch: use the official model card and license; community conversions can differ in chat template and quality.
9. Llama 3.1 8B — best compatibility baseline
It no longer wins raw capability, but Llama 3.1 8B remains a valuable baseline because almost every runtime, accelerator, tutorial, and integration supports it. If a local tool fails with newer architectures, Llama can tell you whether the problem is your stack or the model.
Best for: compatibility testing, fine-tuning education, and stable embedded workflows. Watch: Meta's license is not the same as Apache 2.0, and newer 8B models may deliver better quality.
10. PrismML Bonsai 27B low-bit — best “how does that fit?” experiment
PrismML's Bonsai line uses aggressive low-bit/ternary techniques derived from a larger Qwen-class model, making a 27B model plausible on hardware that ordinary 4-bit math would strain. It is a demonstration that parameter count alone no longer predicts memory.
Best for: low-bit research, reasoning experiments, and 16–32GB enthusiasts. Watch: runtime compatibility, speed, task-specific degradation, and license inheritance. Do not make it your production default until your eval says so.
Ranking table: what actually fits
Rank
Model
Practical quant
Comfortable memory
Use first for
1
GPT-OSS-20B
Q4 on 24GB; lower-bit on 16GB
24GB
Strong general local work
2
Gemma 4 12B
Q4
16GB
Multimodal/general
3
Qwen 3.5 9B
Q4–Q6
16GB
Code, tools, multilingual
4
Ministral 3 8B
Q4–Q6
16GB
Efficient reasoning
5
Qwen 3 8B
Q4–Q6
16GB
Mature local ecosystem
6
Phi-4-mini
Q4–Q8
8GB
Fast utility tasks
7
Gemma 3 4B
Q4–Q8
8GB
Small multimodal
8
MiMo-7B
Q4
16GB
Compact experimentation
9
Llama 3.1 8B
Q4
16GB
Compatibility baseline
10
Bonsai 27B low-bit
Model-specific low-bit
16–32GB
Advanced low-bit testing
Exact sizes vary by conversion. Open the actual GGUF or runtime manifest before downloading; do not budget from parameter count alone.
A safe setup path
For a mainstream Ollama model:
bash
ollama pull qwen3:8b
ollama run qwen3:8b
Then inspect memory while increasing context gradually. For a model not in the official library, use an official or trusted model-card conversion with llama.cpp or import a GGUF into Ollama. Do not copy arbitrary shell installers or model files from unknown mirrors.
Test with five tasks you actually do:
Extract structured data from a messy page.
Summarize a document with citations to passages.
Write or modify code and run a test.
Refuse an unanswerable factual question.
Use a tool schema correctly across ten attempts.
Record tokens per second, peak memory, acceptance rate, and time you spent repairing output. A smaller model at 25 tokens/second can be more useful than a smarter spill-to-RAM model at three.
Quantization reality
Q8 preserves more quality but uses roughly twice the weight memory of Q4. Q4_K_M is a common quality-size compromise in GGUF ecosystems. Q3 and Q2 can make an otherwise impossible model fit, but degradation is not uniform: factual precision, instruction following, coding, and tool syntax may fall differently.
Context also changes memory. An 8B Q4 model may fit comfortably at 8K context and fail or slow badly at 100K because the KV cache expands. Our context-window pricing guide discusses API billing; local context replaces token price with memory and latency.
What a local model is—and is not—for
Choose local for sensitive drafts, offline use, repeatable high-volume small tasks, experimentation, fine-tuning, and independence from service changes. Choose a hosted frontier model when completion quality, huge context, advanced tools, or speed-to-value outweighs control.
Download one model size that leaves memory headroom for the operating system and your application. Record the exact quantization and runtime. Test five short prompts, five long-document prompts, five structured extractions, and five tasks from your actual work. Measure time to first token, generation speed, peak memory, thermal behavior, and whether the output passes your rubric.
Then repeat the same cases with a smaller model and one hosted baseline. A laptop model wins only if it is useful enough, responsive enough, and materially improves privacy, offline access, or cost. “It loaded” is not acceptance. Neither is one impressive answer copied from social media.
Keep the twenty-case set and rerun it when you change quantization, context length, runtime, or hardware acceleration. This small discipline prevents endless downloading and turns local AI into an engineering choice. It also shows whether you need a larger machine, a better model, or simply a narrower task.
Model availability, licenses, runtime support, and quant files were checked July 26, 2026. Memory figures are planning ranges, not guarantees; verify the exact artifact and test your laptop before standardizing a workflow.
