NVIDIA Run:ai Goes Open Source: KAI Scheduler, GPU Fractioning and What You Can Run

NVIDIA has open-sourced the entire Run:ai platform, the GPU orchestration software it acquired and sold as a commercial product, and published it as part of the NVIDIA DSX AI Factory Platform. The announcement on the NVIDIA Developer blog lists the full platform plus four modular components, two related projects, a free community product on NGC, and eight partners integrating it.

If you run AI workloads on Kubernetes and fight over GPUs between teams, this is the most practically useful NVIDIA release this week. It is also easy to over-read, so this post lays out what is in the box, what the licensing looks like, and what NVIDIA does not say.

TL;DR: what is in the release

All repository paths are as listed by NVIDIA in its post.

What problem does Run:ai solve?

GPUs are the scarcest and most expensive resource in an AI team. Default Kubernetes schedules by simple resource counts, which breaks down for AI jobs: a distributed training run needs all of its workers at once or none (gang scheduling), a small inference service wastes a whole GPU if it only needs a slice, and several teams sharing a cluster need guaranteed shares without idle capacity going unused.

Run:ai exists to fix this. NVIDIA describes it as offering "production-grade AI workload and GPU orchestration capabilities" for dynamically allocating GPUs across data processing, training and inference. We covered a similar need from the research side in Ai2's GPU cluster scheduler with time budgets and fair-share, and the cost side in AWS Capacity Blocks GPU price hikes. When GPU hours get more expensive, utilization software gets more valuable.

What does KAI Scheduler do?

A calendar of chips representing KAI Scheduler fair-share and time-based GPU allocation

KAI Scheduler is the engine. NVIDIA says "KAI remains the upstream engine behind NVIDIA Run:ai." According to the post, it provides:

• Gang scheduling with an in-memory prescheduling simulation
• Fair-share and quota allocation, including time-based
• Topology awareness
• GPU sharing through MIG, MPS, time sharing and HaMi
• Support for Dynamic Resource Allocation (DRA) in Kubernetes

It is released under Apache 2.0 and contributed to the CNCF as a Sandbox project, which matters: a neutral foundation home lowers the risk that it becomes a single-vendor dependency.

GPU Fractioning, resource management and Workload Map

The GPU Fractioning Operator brings CUDA-native GPU fractions to Kubernetes with hard GPU memory isolation (it requires CUDA 13.4 fractioning per the post) and a metrics exporter for per-pod visibility. It is scheduler-agnostic, so you could use it without KAI, though it integrates with it.

KAI Resource Management is the organizational layer. It guarantees and shares GPU capacity across teams, for example reserving 50% of a cluster for one group, and targets node pools in heterogeneous hardware.

Workload Map is deliberately narrow: a Go library that describes pods, components and roles so tools treat PyTorchJob, RayCluster or KServe InferenceService consistently for gang and topology-aware scheduling and status. NVIDIA states it does not schedule, queue or autoscale itself.

Model Streamer and Grove

Two related projects ship alongside. Model Streamer reads model weights concurrently and streams them into GPU memory, cutting cold-start latency; NVIDIA says it is integrated into vLLM and SGLang and supported by major clouds and storage vendors. Grove is a Kubernetes API for multicomponent inference workloads with topology-aware autoscaling and hierarchical gang scheduling, including GB200 NVL72 ComputeDomain management. If you serve disaggregated inference, with separate prefill and decode pools, Grove is the piece to read first.

Which component should you adopt first?

The modular design means you do not need to take the whole platform. A rough decision guide, based on NVIDIA's descriptions:

This is our reading of the descriptions, not a benchmark. NVIDIA's post gives numbers only for Model Streamer: Azure benchmarks on eight H100 VMs reported 3x-6x faster loading (about 80 Gbps versus about 40 Gbps for the default path), plus storage-partner results from WEKA, VAST Data and DDN. It publishes no utilization or throughput figures for the scheduler, fractioning or quota components, so any claim about how much they save is something you will have to measure on your own workloads.

What did the commercial product look like before?

Run:ai was an independent company before NVIDIA acquired it, and it was sold as a paid subscription layered on Kubernetes. The scheduler core was already open under the KAI name; what changes now is that the rest of the platform, the control plane and organization layer, is published too. For a platform team, that shifts the build-versus-buy math. Before, getting project-level quotas and a UI meant paying for the product or building your own on top of the scheduler. Now you can read the source, run the community build and decide how much of it you need.

For neoclouds and managed Kubernetes vendors, which are the partners listed, it is also a distribution play: they can offer GPU scheduling as a feature without negotiating a separate license.

What this means for what you build or pay

• If you rent GPUs by the hour, the biggest saving is usually idle capacity. Fractions and fair-share quotas are the two features aimed squarely at that, and with GPU prices moving up this is where scheduler software pays for itself.
• If you run inference services, fractioning plus faster weight loading can let you pack more replicas per GPU and scale from zero with less lag, in principle. Measure before you commit.
• If you manage a shared research cluster, time-based quotas and guarantees reduce the politics of GPU allocation.
• If you are a small team on a single machine, this is overkill. Kubernetes-level orchestration only pays off once there is contention.

Open questions to ask before adopting

1. Which components have independent release cadences and stable APIs, and which are still moving?
2. Is the control plane in the full platform usable without NVIDIA-specific enterprise services?
3. How does GPU fractioning behave with non-NVIDIA accelerators? The post is about NVIDIA GPUs, and nothing suggests otherwise.
4. What is the upgrade path from the community build to a supported enterprise deployment, should you want one?

None of these are answered in the announcement, which is expected for a launch post. Treat the repos' issue trackers and the CNCF Slack channel as your real documentation for now.

Licensing and cost: what NVIDIA says and what it does not

• KAI Scheduler: Apache 2.0, CNCF Sandbox.
• The post says Run:ai source code is governed by its applicable open source licenses and does not name a license for every other component or project. Verify each repo's LICENSE file before you build on it.
• The free NGC Community Product installs via Helm. NVIDIA says "no paid NVIDIA Run:ai subscription or NVIDIA support entitlement is required," but you must accept the NVIDIA Software License Agreement and product-specific terms, and enterprise support is not included.
• For NVIDIA Enterprise AI Factory deployments, Run:ai stays available through commercial offerings such as NVIDIA Mission Control and NVIDIA AI Enterprise.

So "open source" here coexists with a commercial tier. The practical read: community self-service is free; support, SLAs and integrated enterprise packaging are still what NVIDIA sells.

Who is integrating it?

NVIDIA names eight partners: Crusoe (Managed Kubernetes), Mirantis (k0rdent AI), Nebius (one-click deployment on its AI cloud), OpenNebula Systems (OneKS), Rafay (fully managed service), Saturn Cloud (white-label inference via Token Factory), Spectro Cloud (PaletteAI) and vCluster (per-tenant templates). That signals the project is aimed at neoclouds and platform teams, not just large in-house clusters.

How does this fit NVIDIA's DSX strategy?

DSX is NVIDIA's umbrella for the AI factory: hardware, power and software. We covered the hardware side in NVIDIA's OCP Vera Rubin MGX open designs and DSX grid resilience. Opening the orchestration layer is consistent: the more clusters run NVIDIA-friendly open scheduling, the more GPU-hours land on NVIDIA hardware, and the less reason there is to adopt a rival stack. It is also a competitive answer to open-source and cloud-native schedulers that already exist. That reading is ours, not NVIDIA's.

How to try it

1. Read the announcement and pick your entry point.
2. For a low-risk test, install KAI Scheduler alongside your existing scheduler and target one namespace with a gang-scheduled training job.
3. Add the GPU Fractioning Operator if you have many small inference services that waste whole GPUs.
4. Define projects and quotas in KAI Resource Management when more than one team shares the cluster.
5. For full-stack evaluation, install the Community Product from NGC with Helm.
6. Questions and contributions go to #kai-scheduler on CNCF Slack or GitHub issues.

What to watch for

• Maturity. A newly open-sourced commercial platform carries years of enterprise assumptions; expect rough edges in community setups.
• Support. The free path has no NVIDIA support entitlement.
• Lock-in vs. portability. KAI at CNCF is a good sign, but GPU fractioning with CUDA 13.4 ties you to current NVIDIA drivers.
• Security. A scheduler with cluster-wide privileges deserves the same review as any operator you install. See our guidance on agent and infra safety in Anthropic's open-source scanner for maintainers.

Related reading

• Ai2 GPU cluster scheduler
• AWS Capacity Blocks price hike
• NVIDIA OCP Vera Rubin and DSX
• Intel D270 inference GPU
• NVIDIA DGX Spark price and memory

Details reflect NVIDIA's October 13, 2026 announcement; repository contents and licenses may change.