How to Run Kimi K3 Locally on Desktop — Open Weights Prep Guide (July 2026)
Kimi K3 open weights arrive July 27, 2026. Honest hardware tiers for 2.8T MoE, vLLM/SGLang prep, API interim, and K2.7 local fallback — you cannot run full K3 on desktop today.
You cannot run Kimi K3 locally today. Moonshot launched the API and consumer app on July 16–17, 2026 and promised full open-source weights by July 27, 2026 — but as of July 17, there is no public Hugging Face download for the 2.8T model. This guide is a prep playbook: hardware tiers, vLLM/SGLang staging, API interim, and Kimi K2.7 Code as the weights-available-now fallback.
#1 nextjs.org/evals, 1679 Arena Elo — see evals post
July 27 timeline — what to do before weights drop
Date
Action
Now (Jul 17)
Use API for K3 eval on your repos; self-host K2.7 for open-weight coding
Before Jul 27
Stage vLLM cluster, disk space, Hugging Face access, license review
Jul 27+
Download weights from Moonshot HF org; pin hash; run regression eval
Jul 27+ (audit)
Read technical report for training stack; independent groups may probe architecture lineage amid unofficial Claude distillation chatter
Week after release
Expect GGUF quants community ports — verify license and active-param specs
Do not bake unreleased filenames into production Terraform or CI. Pin kimi-k3 API until the model card publishes exact repo IDs.
Distillation note: HN launch threads speculated K3 may imitate Claude/Fable via API harvesting — unverified. Open weights plus Moonshot's technical report are the first chance for third parties to inspect MoE structure and tokenizer overlap; they will not fully prove or disprove synthetic teacher post-training. See the API guide distillation section if provenance affects your deploy policy.
Expected architecture — 2.8T MoE at 1M context
Official platform docs and Moonshot's Kimi K3 tech blog confirm 2.8 trillion parameters — the world's first open 3T-class model — with Kimi Delta Attention (KDA), Attention Residuals (AttnRes), and Stable LatentMoE activating 16 of 896 experts per token. K3 achieves ~2.5× scaling efficiency vs Kimi K2 and uses quantization-aware training from SFT with MXFP4 weights and MXFP8 activations.
On open-flagship scale charts circulating July 17, K3's 2.8T jump from K2 (~1T) is the largest single-generation climb in the open frontier set — ahead of DeepSeek V4 Pro (1.6T) and MiMo V2.5 Pro (1.02T). See the size-over-time chart and comparison table in the API guide. Total params ≠ active params per token — plan hardware from 16/896 MoE sparsity, not the headline 2.8T.
Component
Claim
Stable LatentMoE
16 of 896 experts active per token
KDA + AttnRes
AttnRes kernel optimized 283.6ms → 114.4ms on H200 (Moonshot case study)
MXFP4/MXFP8 QAT
Quantization-aware from SFT — plan quant-friendly serving stacks
vLLM KDA prefill cache
Moonshot contribution ships with weights — watch vLLM release notes
Multimodal
Native vision — local serving may need separate vision tower memory
Exact active parameters per token and memory tables ship with the July 27 model card + technical report. Until then, scale expectations from K2.7 (1T total, 32B active) upward — K3 is a larger frontier tier with higher sparsity.
Kimi Delta Attention — what it means for local inference
Moonshot's Kimi Delta Attention (hybrid linear attention) plus Attention Residuals are optimized for million-token decoding on Moonshot's cloud stack — not automatically portable to your vLLM cluster on day one.
Cloud claim
Local inference implication
6.3× faster decoding at 1M context
Requires kernel/runtime support in vLLM/SGLang builds — community ports may lag weeks
1,048,576-token window
VRAM scales with KV cache size; local max-model-len will start far below 1M unless you shard across many GPUs
Hybrid linear attention
Not all local runtimes implement linear-attention paths on day zero — expect fallback attention = slower + more memory
Attention Residuals
Training-efficiency trick; may need trust-remote-code and pinned Moonshot inference recipes
Practical takeaway: When weights drop July 27, treat 128k–256k local context as the realistic first milestone. Reserve full 1M for API or large clusters until Moonshot (or vLLM upstream) documents a supported local recipe. The API guide remains the right path for million-token doc ingest until local KV economics are measured on your hardware.
Hardware tiers — honest expectations
These tiers are planning brackets, not guarantees. Adjust when Moonshot publishes memory tables.
Tier
Hardware sketch
Realistic K3 use
A — API only
Any desktop
Production today — no local GPU
B — K2.7 local
2–8× A100/H100 or high-end consumer + quant
Open coding now — not K3
C — K3 quant lab
4–8× H100 80GB
Experimental K3 after Jul 27; moderate concurrency
Moonshot documents K2.7 Code with vLLM, SGLang, and KTransformers. For K3, Moonshot contributed a vLLM KDA prefill cache implementation to release alongside weights July 27, 2026 — check upstream vLLM before pinning production images.
Staging checklist (do now)
bash
# Example: ensure vLLM recent enough for large MoE (verify version against K3 model card Jul 27)
python -m venv ~/vllm-kimi-staging
source ~/vllm-kimi-staging/bin/activate
pip install -U vllm
# Disk: reserve multiple TB for bf16 shards + cachedf -h
# Hugging Face token for gated repos if Moonshot gates weights
huggingface-cli login
Expected launch-day serve pattern (illustrative — not real model ID)
bash
# Replace MODEL_ID after July 27 model card — DO NOT RUN TODAY
vllm serve moonshot-ai/Kimi-K3-Instruct \
--tensor-parallel-size 8 \
--max-model-len 131072 \
--trust-remote-code
1M context locally may require reduced max-model-len vs cloud API defaults — cloud benefits from Moonshot's Delta Attention infrastructure. Start eval at 128k–256k context and scale up.
llama.cpp — wait for quants
Community GGUF ports typically follow full weights by days/weeks. For Apple Silicon experiments, watch what is llama.cpp and Qwen 3.6 local patterns — but expect K3 full quants to remain huge even at Q4.
API interim — hybrid with OpenCode
Until weights land, wire K3 as cloud and K2.7 as local in opencode.jsonc:
Kimi K3 local desktop inference is not available July 17, 2026. Moonshot promised open weights by July 27 — prep vLLM/SGLang, multi-GPU capacity, and hybrid API routing now. Use kimi-k3 API for frontier evals; use K2.7 Code for open-weight coding today. Single-consumer-GPU full K3 is unrealistic; plan Tier C/D hardware or stay cloud.
Weight release date and architecture claims accurate as of July 17, 2026 per Moonshot launch messaging. Verify Hugging Face repos and model cards on release day before provisioning hardware.
