SLA: A Vision-Language-Action Model That Adds 3D Sense During Continual Pre-training

Robot policies need language, vision and precise motion. Vision-language-action models have become the usual architecture: a pretrained vision-language backbone plus an action head trained on large robot datasets. The weak spot, argued in SLA (arXiv 2611.08373), is 3D grounding: VLAs often learn geometry too late, mostly from post-training on a narrow task, which limits transfer when scenes shift.

SLA stands for Spatial-Language-Action Model. The authors build on pi0.5-style training and add implicit spatial alignment during continual pre-training, then keep alignment through post-training, while removing auxiliary modules when the policy runs on a real robot. The project page summarizes the story as spatial supervision at foundation scale, not only a late fine-tune trick.

This post explains the training loop, the datasets table on the project site, headline success rates, and what is still unknown from the PDF alone. It is written for readers who already follow our LeRobot and RoboColosseum coverage and want a paper-first map of a new VLA training recipe.

TL;DR

Why 3D alignment belongs in continual pre-training

Most VLAs spend their "foundation" phase on 2D web image–text pairs or on video that compresses depth. Geometry often appears only when you adapt to a robot deck with a modest dataset. The SLA authors argue that waiting until then makes spatial reasoning a bolt-on.

Their hypothesis is simpler: action tokens depend on visual tokens, so if visual tokens encode weak depth, the policy caps out no matter how many language tricks you stack. They show a linear depth-probing experiment in the paper: a frozen VLA's embeddings do not reconstruct meaningful depth until after alignment training, which supports the claim that 2D-only features under-fit 3D structure.

This is the same family of concern we highlighted when comparing policies in RoboColosseum: even strong models can disagree wildly once hardware and scenes change. SLA is one architectural response: shape embeddings early, adapt later.

Continual pre-training versus one-shot fine-tuning

Continual pre-training means more gradient steps on a broad corpus after you already have a strong checkpoint, with care not to erase general features. The paper positions SLA as closer to domain extension than to a tiny LoRA on top of a frozen trunk. For buyers, the distinction matters because foundation competence changes data efficiency on the next robot: policies that internalize geometry tend to need fewer teleoperated hours for a new line, if the paper's downstream tables hold outside the authors' harness.

How SLA trains, in one pass through the loop

The paper breaks the loop into parts that parallel how practitioners think about data planes:

1. Inputs — Multi-view RGB, language instructions, robot state, and when available partial depth maps and camera intrinsics.
2. Frozen 3D teacher — They adopt OmniVGGT as f^{3D} in the manuscript, a visual geometry model that emits pixel-aligned 3D features. It never receives task gradients.
3. Alignment target — Teacher features are resampled to each VLA visual token grid with positional cues so tokens match spatially.
4. Multi-layer gates — Instead of picking one transformer block by hand, SLA learns feature-dependent gates across layers so different robot corpora can emphasize different depths in the stack. That matters because simulators and labs paint textures and lighting differently; a single "best layer" rarely fits all.
5. Action loss plus alignment loss — Standard action prediction stays on; a cosine alignment term pulls visual tokens toward the teacher. The paper combines them with a scalar weight.
6. State dropout — Randomized dropout on privileged state pushes the policy to lean on visual cues, reducing trajectory memorization when demonstrations repeat structure.

At deployment, the manuscript stresses that teacher, projector and gates are training-only; the runtime policy is still the pi-class action head you would ship today.

Three columns of linked nodes evoke layered network activations fused for spatial supervision

Continual corpus scale (project table)

The SLA project page publishes a source table for continual pre-training with rounded episode, task and hour counts drawn from public datasets such as InternData-A1, AgiBotWorld-Beta, RoboCasa365-Pretrain, RoboCOIN, DROID and MolmoAct. Those numbers describe source statistics before curation, not guaranteed SLA training tokens, so use them as order-of-magnitude coverage, not as a reproducibility recipe by themselves.

The table matters for two reasons. First, it shows the authors merged conventions across arms and scenes, which is the painful part of cross-embodiment training. Second, it underlines data diversity as a precondition for their alignment objective: a geometry teacher is only useful if batches actually contain varied camera poses and contact-rich motion.

Reported benchmark highlights

The following figures are claims from the paper's introduction and results tables (the abstract itself gives only the 24.0-point real-world gain). They are not re-measured by explainx.ai.

• LIBERO-Plus — 87.4% weighted success across seven perturbation dimensions (pi0.5: 84.4%).
• RoboTwin 2.0 — 72.3% in Clean2Random and 79.2% average across Clean2Clean (86.0%) and Clean2Random.
• RoboCasa-Panda — 68.4% with 50 downstream demonstrations per task, compared with some baselines that used 300 to 3,000 demos per task.
• RoboDojo-Real — 33.9% overall success across three embodiments versus 12.8% for pi0.5. The paper adds a memory mechanism in post-training for this benchmark.
• Eight real conditions — 24.0 percentage-point average gain over pi0.5 (83.8% average success) on the paper's private deployment conditions.

Read these as paper-reported scores with the usual caveats: different trial counts, seeds and filtering can move percentages, and real robots add breakage modes simulations skip.

Why heterogeneous geometry supervision matters

Robot datasets disagree about calibration quality. Some logs ship accurate intrinsics; others only have approximate camera models; many lack depth entirely. SLA's training-time use of partial geometric labels is an attempt to consume what exists instead of dropping those trajectories. That is aligned with the field's push toward imperfect but large corpora rather than perfect but tiny ones.

How SLA compares with explicit 3D fusion

Prior lines such as PointVLA and GeoVLA (cited in the paper) inject point clouds or fused tokens at inference. That can work well in labs with depth cameras but raises deployment cost and calibration fragility. SLA's pitch is representation-level 3D: learn geometry inside RGB pathways you already ship.

The tradeoffs are:

• Heavier training — You pay for the teacher forward pass and multi-layer gates during development.
• Teacher bias — If f^{3D} is wrong in a niche domain, alignment can encode the same blind spot.
• Evaluation honesty — Strong sim scores must still meet real-site evals; overfitting to renderer quirks remains a risk.

What to download if you want to reproduce

According to the project page, artifacts land in multiple places:

• Paper and proofs — arXiv abs page.
• Code — Linked under Spatial-Forcing/SLA, sharing lineage with the Spatial Forcing line of work the authors cite.
• Weights — Hugging Face collections such as haofuly/SLA_base appeared in discovery tools; the model card was empty when we opened it, so treat checkpoints as preview until README instructions stabilize.

If you only need design patterns, read the method section: the gated layer fusion and dropout schedule are the portable ideas even when weights are gated.

Shelves of clustered vectors suggest an organized corpus of robot experience feeding foundation training

What this means for teams picking robot stacks

1. Ask about geometry — If a vendor says "VLA," ask whether 3D supervision was pre-training, post-training or none. SLA is evidence that where you inject spatial loss changes scaling.
2. Budget compute honestly — Continual pre-training with a second frozen model is not cheap; the paper mentions roughly 110 hours on 48 H100 GPUs for the continual stage in one configuration, which is a planning number for internal clusters.
3. Plan evaluation suites — Combine sim breadth with real disturbance tests. Our Verne Robotics v0.5 write-up walked through LIBERO versus Robotwin tradeoffs for open-weight users; SLA sits adjacent as a training recipe, not a product SKU. If you need hosted fine-tuning instead of owning a cluster, compare against Physical Intelligence's pi API beta and the dataset scale in FineART on LeRobot.

Open questions

• Licensing — Mixed public corpora mean downstream commercial use depends on each dataset's terms; the paper is not a lawyer.
• Cross-embodiment honesty — Strong averages can hide task clusters that fail silently; demand per-task tables when negotiating a vendor bake-off.
• Teacher dependence — If OmniVGGT-class models refresh, how stable are SLA checkpoints? The paper does not answer versioning policy.

Related reading on explainx.ai

• RoboColosseum: real-robot evaluations across policies
• Uni-robot-agent and RoboDojo harness lessons
• Ai2 ManiFlow-TB and ToyBlocks as a controlled VLA testbed
• LeRobot v0.7.0: datasets, streaming and new policies
• Verne Robotics v0.5: open-weight world-action model claims

Sources: arXiv 2611.08373, SLA project site, OpenHelix Spatial-Forcing SLA directory

Benchmark figures follow the November 2026 manuscript; check the latest arXiv revision for corrections.