LingBot-Map: Streaming 3D Reconstruction at 20 FPS — Robbyant GCT Guide (2026)
Robbyant lingbot-map (13.1k★) is a feed-forward 3D foundation model — Geometric Context Transformer, ~20 FPS at 518×378, 10k+ frames, FlashInfer KV cache. explainx.ai install, checkpoints, windowed mode, and vs SLAM/VGGT.
3D ReconstructionComputer VisionPhysical AIOpen SourceRobotics
LingBot-Map from Robbyant Team is one of the most-starred open 3D stacks of mid-2026: 13.1k GitHub stars, 1.4k forks, Apache 2.0. It is a feed-forward 3D foundation model — you stream frames in, you get poses + dense geometry out, without running COLMAP-style optimization for every new scene.
The pitch in the README is concrete: ~20 FPS at 518×378 on sequences beyond 10,000 frames, using a Geometric Context Transformer (GCT) and paged KV cache via FlashInfer.
For builders in physical AI — robots, world models, AR — LingBot-Map sits between classic SLAM (slow to tune, brittle) and per-scene NeRF/GSplat (quality but not real-time streaming).
Feature matching + BA (COLMAP, ORB-SLAM) — accurate but heavy per scene
Neural fields (NeRF, 3DGS) — great visuals, offline training per location
Feed-forward transformers (VGGT lineage) — one network forward pass generalizes across scenes
LingBot-Map targets (3) for streaming:
GCT component
Role
Anchor context
Coordinate grounding — where am I in the map?
Pose-reference window
Local geometric cues from recent frames
Trajectory memory
Long-range drift correction without resetting state every frame
Output per frame: camera pose, depth / point cloud suitable for viser visualization or offline MP4 flythroughs.
Unlike iterative SLAM, there is no explicit loop-closure optimizer you configure — the transformer is supposed to internalize drift control. The loop example scene in example/loop stress-tests that story.
Open http://localhost:8080 — browser-based viser point cloud.
Checkpoints — which .pt to pick
Checkpoint
Use when
lingbot-map-long
Long sequences, large-scale outdoor (Oxford, drive scenes)
lingbot-map
Paper/benchmark default; balanced short + long
lingbot-map-stage1
Load into VGGT for bidirectional c2w inference
Robbyant notes a stronger long-sequence model is training — watch the repo News section.
Example scenes (shipped in repo)
Folder
Notes
example/courthouse
Outdoor — use --mask_sky
example/university
Campus scale
example/loop
Loop-closure trajectory
example/oxford
Large outdoor — sky mask
Extra demo sequences: robbyant/lingbot-map-demo on HuggingFace (travel, indoor MP4, LingBot-World clips).
The 320-frame KV cache limit (read this before long video)
Training uses video RoPE on 320 views. If the KV cache holds more than 320 views, quality degrades.
Mitigations:
Flag
Effect
--keyframe_interval 2 (etc.)
Only every Nth frame stored in cache; others still predict
--mode windowed
Sliding windows for sequences over 3000 frames
--window_size 128
KV slots per window (not raw frames — see README math)
--overlap_keyframes 8
Cross-window alignment when keyframe_interval > 1
June 28, 2026 fix: SDPA KV cache bug — long sequences work better on SDPA now; FlashInfer still preferred.
April 24, 2026 fix: FlashInfer was silently caching non-keyframes when keyframe_interval > 1 — pose quality past 320 frames should improve after pulling latest main.
If you see pose collapse on very long trajectories, README says switch --mode windowed — often keyframe_interval tuning alone is enough.
Camera paths are YAML-driven (demo_render/config/indoor.yaml, outdoor_drive.yaml) — follow, birdeye, static, pivot segments without re-running inference.
Sky masking (--mask_sky) downloads skyseg.onnx from HuggingFace on first use — caches masks beside image folders. Critical for outdoor point clouds.
Benchmarks shipped in repo
Robbyant released eval scripts for:
Dataset
Folder
KITTI
benchmark/
Oxford Spires
preprocess/oxford.py first
ETH3D, TUM, 7-Scenes
scripts added 2026
Tanks and Temples, NRGBD
scripts added 2026
VBR, Droid-W
scripts added 2026
Claim: SOTA vs streaming and iterative optimization baselines on diverse benchmarks (see paper PDF in repo). For robotics sim, compare against NVIDIA Cosmos 3 world models and Xiaomi U0 synthetic data — different layer (generation vs metric reconstruction).
LingBot-World loop — generated video → 3D map
Robbyant runs batch_demo.py on LingBot-World synthetic footage (lingbo_world_frames.mp4) with the same flags as outdoor drive scenes.
That matters for 2026 world-model stacks:
text
Text / action → LingBot-World (video)
↓
LingBot-Map (geometry + pose)
↓
Point cloud / flythrough / NPZ exports
Real robots still need teleop datasets like HIW-500 — but for sim-to-preview and content pipelines, feed-forward reconstruction removes a COLMAP step from synthetic video.
GPU + CUDA stack; 320-view RoPE limits; pose collapse on extreme length
COLMAP / SLAM
Mature, metric, no GPU model
Per-scene tuning; not 20 FPS dense recon
3D Gaussian Splatting
Visual quality
Offline per scene; not designed for 10k-frame streaming
VGGT
Strong feed-forward baseline
LingBot-Map extends with GCT streaming + KV cache story
Cosmos / gen world models
Physics-rich futures
Not a drop-in metric map from real RGB stream
LingBot-Map is not a replacement for lidar SLAM on a factory floor today. It is a credible open stack for turning long RGB streams into explorable geometry fast enough to feel live.
Reconstruct LingBot-World outputs to validate consistency
Indie / AR creator
Courthouse demo → custom video → MP4 flythrough
Production lidar mapping
Wait — verify metric accuracy on your domain first
Summary
LingBot-Map (13.1k★, Apache 2.0) is Robbyant's Geometric Context Transformer for streaming 3D reconstruction — ~20 FPS feed-forward inference, FlashInfer KV cache, viser interactive demo, and batch_demo.py for 25k-frame flythroughs. Use lingbot-map-long for long outdoor runs; respect the 320-view RoPE limit via keyframe_interval or windowed mode. Benchmarks cover KITTI, Oxford, ETH3D, TUM, and more; LingBot-World videos reconstruct through the same pipeline. For physical-AI builders, it is the open geometry layer next to world models and robot teleop data.
Install paths, star counts, and benchmark list accurate as of July 19, 2026 per public README. PyTorch/Kaolin/FlashInfer versions change — verify repo News before production deploys.
