WorldFoundry: Open Infrastructure for World Models and Benchmarks

WorldFoundry is a new open stack for running and scoring world models — systems that keep persistent state, react to actions, and roll forward coherent trajectories, not just one-off video clips. The team published arXiv:2611.10771 on November 7, 2026 and ships code at OpenEnvision/WorldFoundry under Apache-2.0.

If you benchmark embodied agents, navigation world models, or long-context video synthesis, WorldFoundry is meant to replace ad-hoc scripts with one conda pipeline, model zoo manifests, and evidence-aware scorecards.

TL;DR

How WorldFoundry defines a world model

The paper argues that pretty video is not enough. A generative world model must:

1. Maintain a history-conditioned representation of an environment.
2. Update state when agents or users intervene.
3. Emit temporally coherent futures that respect counterfactual actions.

That definition sits above raw text-to-video tooling because it demands closed-loop behavior — the same bar robotics foundation work chases in simulators.

WorldFoundry's intelligence-oriented framework spans:

• Perceptual grounding
• Persistent state
• Controllable transformation
• Dynamics-aware prediction
• Closed-loop interaction

Readers comparing to game engines should see WorldFoundry as evaluation plumbing, not a replacement for Unity or Omniverse — closer to an agent harness for world simulators.

What is in the repository

The GitHub README organizes the project into surfaces builders actually touch:

Integrated systems named in release notes include Wan, HunyuanVideo, LTX2, Cosmos, Matrix-Game-2/3, LingBot world and VLA stacks, OpenPI, OpenVLA-OFT, and GROOT — a breadth that mirrors how NVIDIA GR00T demos mix video and action policies.

Layered map pins suggesting multi-benchmark world-model evaluation geography

Install path: clone to conda

WorldFoundry assumes conda, not pip-only laptops. The documented happy path:

git clone https://github.com/OpenEnvision/WorldFoundry.git
cd WorldFoundry
bash scripts/setup/bootstrap_worldfoundry.sh
source tmp/worldfoundry_unified_env.sh
conda activate "${WORLDFOUNDRY_UNIFIED_ENV_PREFIX}"

Large demo media lives in Git LFS. For a faster clone:

GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/OpenEnvision/WorldFoundry.git

Checkpoints and datasets stay outside git — point --data-root, --model-root, and --artifact-root at shared disks when you are on a lab NFS volume.

CUDA wheels pin by host capability (cu128 default on CUDA 12.8). Older hosts can pass --cuda cu124 or cu121 per README.

After bootstrap:

worldfoundry-eval --help
worldfoundry-eval zoo models --json
worldfoundry-eval zoo benchmarks --json

This mirrors how Samir's vLLM-from-scratch guide insists on reproducible env files before you trust benchmark numbers.

First inference: matrix-game-2

The README recommends matrix-game-2 (Skywork/Matrix-Game-2.0 on Hugging Face) as a navigation-style starter:

bash scripts/inference/prepare_model_infer.sh matrix-game-2 --download
worldfoundry-eval zoo model-download --model-id matrix-game-2 --check-local --json
bash scripts/inference/test_nav_video_gen.sh matrix-game-2 \
  --output-dir tmp/matrix_game2_first_run

Outputs should include media, logs, and manifest metadata under the output directory. Treat a clip as evidence only after you visually confirm it matches the prompt and runtime profile.

For interactive exploration:

python -m pip install -e ".[tui]"
worldfoundry-eval tui

The TUI prints a runnable CLI before expensive GPU jobs — useful when you are comparing against Helo agent primitives on the same workstation.

Studio workspace

Launch the browser UI from the unified env:

bash scripts/workspace/run_workspace.sh \
  --host 127.0.0.1 \
  --port 7870 \
  --max-jobs 8

Open http://127.0.0.1:7870/, configure jobs under Create Job, and use Visualizers for World/Gradio, Viser, or embodied bridges. Remote users must forward 7870 and any viewer ports.

Studio is the intended path for release audits — model-specific forms, previews, and artifact links in one place instead of scattered bash logs.

Evaluation and scorecards

WorldFoundry separates running models from claiming leaderboard ranks.

Example official run:

worldfoundry-eval run \
  --model matrix-game-2 \
  --benchmark vbench \
  --mode official-run \
  --output-dir tmp/hello_world_run \
  --json

Inspect:

• run_manifest.json — model, benchmark, timestamps
• results.jsonl — per-sample records
• metrics/summary.json — aggregates and skip counts
• scorecard.json — readiness, eligibility, blockers

The paper stresses evidence-aware scorecards: a metric is valid only when artifacts match benchmark layout and official runner constraints. Import-only validation exists for teams that already generated videos elsewhere:

worldfoundry-eval zoo benchmark-run \
  --benchmark-id vbench \
  --mode official-validation \
  --official-results-path <official_results.json> \
  --generated-artifact-dir <generated_videos> \
  --output-dir tmp/benchmark_zoo/official_validation/vbench \
  --json

That design matches lessons from Raindrop's Tau2 bench reporting: publish traces, not just percentages.

Before you tweet a number, run integrity commands:

worldfoundry-eval run --plan-only --fail-on-overclaim --fail-on-stale --json

Benchmark catalog highlights

The Hugging Face paper page and README reference benchmarks including VBench, VideoScore, WorldScore, WorldModelBench, Physics-IQ, T2V-CompBench, WorldReasonBench, and embodied suites wired through Docker VLA harness patterns.

WorldFoundry's value is normalizing those disparate evaluators behind one CLI — similar in spirit to AllenAI's VLA evaluation harness cited in acknowledgments, but extended to non-embodied world video.

Compare readiness flags in the benchmark zoo before you schedule multi-day GPU jobs — the catalog records execution evidence separately from marketing claims, a pattern more labs should copy after claimed-vs-verified benchmark disputes.

Who should adopt this now?

If you only need a single SAM-3 segmentation demo, WorldFoundry is oversized. If you compare ten world models weekly, it may save days.

Limitations and active development

The README's warning is blunt: "still under active development." Expect:

• Catalog entries with metadata-only stubs
• Models that need per-model conda splits
• Demo media that does not substitute for full benchmark assets
• Documentation drift between English and Chinese Fumadocs routes

Do not treat WorldFoundry star counts or HF paper upvotes as maturity proxies — verify your model row's runtime profile.

How WorldFoundry relates to other world-model stacks

For NVIDIA-centric robot training narratives, see Crossnokaye AgentCore physical AI — enterprise orchestration, not academic scorecards.

What people are asking

Is there a pip-only install?
No — conda bootstrap is the supported OSS path.

Can I use API-only models?
The catalog includes hosted API entries with metadata-only rows; you still need keys and rate-limit discipline.

Does it train models?
README v0.2.0 notes emphasize inference refactors; training integrations appear as upstream modules, not one-click fine-tunes.

How do I contribute?
Port inference into worldfoundry/, declare checkpoints in the catalog, run the smallest official demo, and attach evidence in docs — same bar as Janestreet agent skills contribution hygiene.

Reading scorecards without overclaiming

When scorecard.json lands in your output directory, treat it like an audit packet, not a marketing badge. WorldFoundry documents several fields practitioners should read before citing a number in a deck:

The project’s --fail-on-overclaim and --fail-on-stale flags exist because world-model papers often mix demo clips with full-suite scores. A model that runs on a cherry-picked prompt may still show blockers for Physics-IQ or embodied suites that need Dockerized simulators.

Before you compare against Raindrop's Tau2 numbers or Opus formalization scores, archive the run manifest hash alongside the scorecard. Repro disputes settle faster when artifacts are addressable.

Concentric audio rings suggesting multi-modal spatial sensing in world simulators

Maintainer checks the README expects

Open-source contributors run a fast core test slice before promoting catalog entries:

PYTHONPATH=. python -m compileall -q worldfoundry scripts
PYTHONPATH=. python -m pytest -m fast_eval_core test/eval_core
bash scripts/docs/build.sh --skip-bootstrap

That bar is higher than dropping a Gradio demo — closer to how Janestreet ships agent skills with executable tests. If you only consume WorldFoundry, you still benefit: promoted models passed at least the fast_eval_core gate on maintainer machines.

Hardware planning for shared clusters

WorldFoundry’s README calls out NVFP4, FlashAttention 2/3, SageAttention, and multi-GPU sequence parallel paths. On a shared university cluster, set artifact roots on NFS so teammates do not duplicate Hugging Face snapshots:

bash scripts/setup/link_hf_checkpoints.sh \
  --ckpt-dir "${WORLDFOUNDRY_CKPT_DIR}" \
  --hfd-root "${WORLDFOUNDRY_HFD_ROOT}" \
  --hf-hub-cache "${HF_HUB_CACHE}" \
  --default-world

Queue schedulers should pin GPU compute capability explicitly — the bootstrap script maps hosts to cu128 vs older wheels. Mixing driver versions on one benchmark matrix invalidates latency comparisons, the same lesson as Samir’s vLLM build guide stresses for inference micro-benchmarks.

Policy and safety angles

World models trained on internet video inherit safety and copyright risks separate from LLM chat policies. WorldFoundry does not replace a red-team harness — pair it with NVIDIA Open Agent Safety Platform thinking when actions in sim can drive real robots.

For embodied policies (OpenVLA, GROOT, Octo adapters listed in release notes), log action bounds and episode seeds in manifests so incident reviews can replay trajectories — the same discipline as Berkeley–NVIDIA CORL embodied agent work.

Run the documentation site locally

cd docs/fumadocs
npm ci
npm run dev -- --port 8014

Build static docs from repo root:

bash scripts/docs/build.sh

English routes live under /docs; Chinese under /zh/docs.

Related reading

• LongBanana long-context image synthesis benchmark
• Open HumanScale embodied data release
• Skild Steel Solver and physics sims
• Agentic robotics CORL research
• Multimodal vision token budgets
• Beneath the Surface agent harness evolution
• ByteDance Seedream editing rounds

Sources

• arXiv:2611.10771 — WorldFoundry paper, November 7, 2026.
• OpenEnvision/WorldFoundry GitHub repository, README and catalogs accessed November 10, 2026.
• Hugging Face papers entry 2611.10771, November 2026.

Catalog counts and integrated model lists change with releases; verify worldfoundry-eval zoo output on your pinned commit before citing numbers in production reports.