NVIDIA has published a hands-on guide to running Kubernetes on Vera Rubin NVL72, built around its open-source AI Cluster Runtime (AICR). The NVIDIA Technical Blog post, dated November 2, 2026 and written by Mark Chmarny and Yuan Chen, says a Vera Rubin recipe family passed deployment, conformance and a first cross-node performance gate, with 744.71 GB/s of NCCL all-reduce bus bandwidth at 16 GiB over NVLink.
This is early-silicon guidance, and NVIDIA says so: "It is still early days for Vera Rubin." The recipes ship as Preview. This post summarizes what changes from GB200 NVL72, what NVIDIA validated, what it did not, and how to try the recipe generator.
At a glance
| Question | Answer from NVIDIA's post |
|---|---|
| What is AICR? | NVIDIA AI Cluster Runtime, part of the DSX AI Factory Platform |
| What does it do? | Captures tuned configs as recipes, renders deployment bundles, validates live clusters |
| Which hardware? | Vera Rubin NVL72 compute trays (accelerator tag vr200) |
| Which stack was validated? | GPU Operator v26.7.0, DRA driver v0.5.0, R615.23 driver, Ubuntu 26.04 (64k-page kernel), RKE2 |
| Kubernetes floor | 1.34.1 or later |
| Headline result | 744.71 GB/s NCCL all-reduce bus bandwidth at 16 GiB, across two trays over NVLink |
| Status | Preview, not a Supported recipe |
| Not yet validated | Inference serving performance, MIG, RDMA data-path over ConnectX-9 |
What changes from GB200 NVL72?
NVIDIA calls Vera Rubin NVL72 "an evolution of the same NVIDIA MGX rack-scale architecture, now in its third generation," but lists several user-visible changes for teams planning an upgrade:
- GPU driver. A new driver branch. NVIDIA validated R615.23, a host-installed open GPU driver. Early support is targeted for R610 and official platform support for R620. AICR's GB200 baseline is R580.173.02. NVIDIA stresses that R615.23 is "the current tested version, not a confirmed minimum."
- NICs and DPUs. ConnectX-9 SuperNICs and BlueField-4 DPUs replace ConnectX-7 and BlueField-3. DOCA 3.3.0 is the first GA release supporting ConnectX-9.
- Topology. Both reference trays have two CPUs and four GPUs, but the Vera Rubin tray exposes eight ConnectX-9 PCI device groups with 32 functions, against four ConnectX-7 adapters on the DGX GB200 NVL72 tray. Check GPU-to-NIC affinity and NUMA placement for RDMA and GPUDirect Storage workloads.
- IMEX. With DRA-managed IMEX, stop, disable and mask the host's static nvidia-imex service to prevent ownership conflicts. DRA v0.5.0 adds an alpha host-managed option with domain isolation only.
- MIG. Rubin GPUs introduce new MIG profiles, but the AICR recipes use whole-GPU allocation and do not configure MIG.
If you run GB200 today, our earlier posts on the Vera Rubin NVL72 claims and the DSX Pod containerized Vera Rubin factory give the hardware context.
How does AICR work?
AICR turns a cluster setup into three steps, all shown in NVIDIA's post with real output from a live Vera Rubin reference cluster running the AICR v1.0.0 toolchain.
- Recipe.
aicr reciperesolves criteria (service, accelerator, OS, intent) against the catalog. For rke2, vr200, Ubuntu and training it produced a recipe with 14 components from five overlays, in seconds and offline. - Bundle.
aicr bundlerenders ordered, per-component Helm charts with one deploy script, 61 files in the example. - Snapshot and validate.
aicr snapshotcaptures actual cluster state, thenaicr validateruns checks as in-cluster jobs across deployment, conformance and performance phases.
The 14 components include cert-manager, DraNet, Node Feature Discovery, the NodeWright operator and customizations, the GPU Operator, KAI Scheduler, the NVIDIA DRA driver, NVSentinel and the Prometheus stack. KAI Scheduler is the open-sourced Run:ai scheduler we covered in NVIDIA's DSX AI Factory post.
What did NVIDIA validate?
AICR recipe validation dashboard for the vr200 rke2 ubuntu training recipe, with Kubernetes Vera Rubin deployment, performance and conformance tests passing
Image: NVIDIA
NVIDIA reports that both the training and inference recipe variants validate end to end for deployment and conformance. For training, the result table shows these passes:
| Phase | Checks | Notes |
|---|---|---|
| Deployment | operator health, GPU Operator version, expected resources, nvidia-smi | 4 of 4 passed |
| Conformance | platform health, operator health, accelerator metrics (DCGM), AI service metrics, DRA support, gang scheduling, pod autoscaling, secure accelerator access | 8 of 8 passed |
| Performance | nccl-all-reduce-bw-nvls | 744.71 GB/s bus bandwidth at 16 GiB across both compute trays |
A two-node ComputeDomain test went from apply to two running pods, one per tray, with IMEX channels injected, in about 12 seconds. The full recipe, bundle, snapshot, deployment and conformance sequence took about five minutes on a live cluster, and the performance phase adds the NCCL job's own run time.
NVIDIA also says the first deployment targeted a combination that none of these components had been validated on together (arm64, Ubuntu 26.04, a 64k-page kernel, RKE2 and Vera Rubin), and reached a healthy runtime "in hours." A second cluster reproduced it the next day.
The 744.71 GB/s number in context
The bandwidth figure is a gate: the validator checks a cross-node all-reduce over NVLink through an IMEX ComputeDomain against a bandwidth floor and passes or fails. NVIDIA calls it "the first performance validation on Vera Rubin silicon." It ran on two compute trays on early hardware. It is not a peak-rate benchmark, and NVIDIA does not publish the floor it used in the text we read. Do not compare it with rack-level or multi-node numbers from other posts. For independent cost and throughput comparisons, see our SemiAnalysis Rubin NVL72 coverage.
The stack decisions encoded in the recipe
NVIDIA says the recipe captures choices that teams would otherwise rediscover:
- Host-managed driver. The Vera Rubin image pre-installs the arm64 open-kernel-module driver, so the GPU Operator runs with
driver.enabled: falseand the container toolkit stays operator-managed. The recipe moves to the operator-managed driver once official builds ship. - DRA with ComputeDomains. IMEX-backed ComputeDomains for the NVL72 fabric, with whole-GPU allocation on the device plugin.
- Kubernetes 1.34.1 or later. 1.34 for the GA resource.k8s.io/v1 DRA API, and the .1 patch because the RKE2 CDI/NRI toolkit path needs containerd 2.1.
- Ubuntu 26.04 with the Vera-optimized 64k-page arm64 kernel.
- Small sharp edges. The Node Feature Discovery topology updater's default 60Mi memory limit gets OOM-killed on multi-terabyte NVL72 workers, so the recipe raises it.
- A fresh-install note. GPU Operator v26.7.0 ships a stricter ComputeDomain CRD that requires
numNodeswith no default, so set it explicitly on new clusters.
What is not validated
NVIDIA is open about the limits:
- Inference serving performance is deferred. vLLM merged Rubin (sm_107a) support in v0.27.0, but the kernel path needs CUDA 13.4 or later at build time, and vLLM's published images through v0.28.0 still build against CUDA 13.0.3. ai-dynamo's latest release pins an older vLLM. NVIDIA says a floor needs a published, CUDA 13.4-built runtime image "verified on real hardware."
- MIG on Vera Rubin is not covered.
- RDMA data-path performance over ConnectX-9 is not part of this validation.
- Network Operator compatibility with the new NICs is a separate qualification step.
- Support level. The recipes are Preview: "an early-adopter path that does not yet carry the production support and lifecycle qualification of a Supported recipe."
NVIDIA also notes early-silicon friction. One early cluster had a fabric issue that kept a compute tray out of NVLink until it was reseated. And validation found an unmapped Rubin architecture value in go-nvlib that crash-looped the DRA driver's kubelet plugin. The fix took about three days from first crash report to a validated build and shipped in DRA driver v0.5.0 (go-nvlib 0.11.0 to 0.12.0).
How to try it
The recipes are in the open-source NVIDIA/aicr repository, v1.0.0. The generator commands from NVIDIA's post are:
aicr recipe --service rke2 --accelerator vr200 --os ubuntu --intent training
aicr recipe --service rke2 --accelerator vr200 --os ubuntu --intent inference --platform dynamo
Generating a recipe is offline, so you can inspect the resolved components and versions without owning Vera Rubin hardware. NVIDIA also publishes signed validation evidence for the training recipe (three phases) and the inference recipe (deployment and conformance). Because the recipes are Preview and still moving, treat that evidence as a record of the version it was captured against, since a digest check against today's recipe will report a mismatch.
Why this matters
For platform teams. The cost of a new GPU generation is often the weeks spent matching drivers, kernels, runtimes and schedulers. NVIDIA's pitch is that a versioned recipe plus signed evidence cuts that to hours. The claim is NVIDIA's own and the cluster counts are small (two), but the approach, publishing failing edge cases such as the NFD memory limit, is useful even if you never touch AICR.
For cloud buyers. Vera Rubin capacity is arriving in stages. See Google Cloud's A5X Vera Rubin NVL72 availability and CoreWeave's first multi-rack cluster. Whether a provider's Kubernetes layer is validated against these requirements will matter for how quickly you can run jobs.
For people running GPU clusters now. NVIDIA's Mission Control 2.5 autonomous recovery and CUDA 13.4 GPU fractioning guide cover adjacent operational pieces. The CUDA 13.4 dependency is the same one that blocks the inference-performance validation.
What to watch
- AICR moving the Vera Rubin recipes from Preview to Supported.
- A published, CUDA 13.4-built vLLM or Dynamo image, which would unlock inference performance validation.
- The R610 and R620 driver branches NVIDIA names for early and official platform support.
- Independent cluster operators reproducing the result on non-NVIDIA reference environments.
Related reading
- Vera Rubin NVL72 claims: one-fourth the GPUs for a 10T MoE
- NVIDIA DSX Pod: containerized Vera Rubin AI factory
- NVIDIA open-sources Run:ai KAI Scheduler
- Google Cloud A5X Vera Rubin NVL72 general availability
- CoreWeave first multi-rack Vera Rubin cluster
- NVIDIA Mission Control 2.5
- Sources: NVIDIA Technical Blog, NVIDIA/aicr on GitHub
Versions and results are accurate as of November 2, 2026. Recipes are Preview and change quickly; check the repository.