Nvidia Opens Run:ai, Giving Companies More Ways to Share Their GPUs

NVIDIA has released Run:ai, software that assigns AI jobs to graphics processors, as an open-source platform. Announced October 13, the release lets companies inspect, modify and deploy the system, or use selected components inside an existing setup. For teams operating expensive shared computing equipment, it changes how they can obtain the software and who they can ask to maintain it. NVIDIA’s announcement

Run:ai sits above the machines that train and serve AI models. It helps decide which team gets capacity, where a job runs and how unused resources can be shared. Making that code available reduces one purchasing barrier. A company still needs staff or a provider to keep the underlying systems working and decide how competing users receive access.

NVIDIA’s installation documentation makes the division unusually clear. Run:ai runs on infrastructure the customer owns and operates, including its hardware, Kubernetes cluster and network. Kubernetes is the widely used software that starts and manages application containers across machines. Run:ai adds GPU-aware management to that environment. Installation overview

A platform, or selected parts

The main repository includes the control plane, cluster services, command-line tools, installation code and local development support. Its README describes both a complete platform and modular adoption. It also acknowledges that for a single team with dedicated GPUs, “native Kubernetes scheduling may be sufficient.” Run:ai repository

That qualification is useful. A team whose existing setup already assigns its machines adequately may gain little from installing another management system. A larger organization sharing capacity among research groups, production services and temporary experiments faces a different problem: rules about who waits, who can borrow idle capacity and who can interrupt a running job.

The release allows those organizations to choose components rather than replace everything together. NVIDIA describes separate tools for scheduling, dividing GPU resources, managing organizational allocations and describing the structure of workloads. It says the GPU Fractioning Operator enforces each workload’s memory limit through the driver. NVIDIA’s release details

KAI Scheduler handles decisions such as placing related jobs together, using available machine space and distributing capacity among queues. Its documentation separates a job’s priority from whether it can be interrupted. Those are different decisions for a production service that must remain available and an experiment that can be resumed later. KAI Scheduler documentation

The scheduler also supports placing a group’s workers together, often called gang scheduling. Starting only some workers in a distributed training job can reserve equipment without letting training proceed. A policy that waits until the required group fits can avoid that particular form of wasted allocation. It can also make a large job wait while smaller ones fit, which makes the organization’s allocation rules important.

Another component, Workload-Map, describes where to find roles, replicas and status inside different Kubernetes workload types. That gives other tools a common way to inspect a job without separately hard-coding every format. It is a description and integration layer; the repository does not present it as a standalone scheduler or queue. Workload-Map repository

For a company already using Kubernetes, selecting one of these parts can limit the scope of an initial trial. It can also divide responsibility across more software projects. The useful question is which specific failure an added component addresses and which team will own that component when an upgrade breaks its integration.

Open code and supported operation

NVIDIA offers an open-source release with community support and a support-entitled release for customers with the appropriate NVIDIA support arrangements. The installation guidance distinguishes those paths. A downloaded community installation does not include the same service commitment as a paid support agreement. Run:ai release and support options

The repository’s support policy names a current supported release branch and directs usage questions to GitHub Discussions, bugs to GitHub Issues and private security reports to NVIDIA’s security team. That gives a company routes for reporting problems. It does not, by itself, promise a resolution deadline for an outage. Repository support policy

The operator also remains responsible for maintaining its cluster, monitoring services and restoring backups. Installation requires decisions about storage, network access, certificates and identity management. Some components can be installed with guided tools, but an organization still needs an owner for their operation. Installation responsibilities

This creates several possible commercial arrangements. A company can run community software itself, purchase an eligible supported installation or hire a provider to operate a managed environment. The price comparison should include staff time and the scope of support, as well as software charges. The release announcement alone cannot establish which arrangement will cost less for a particular cluster.

The repository uses Apache 2.0 for its code, with notices and exceptions for included third-party material and branding. Organizations can inspect and adapt the licensed code without assuming that every image, name or bundled dependency carries identical rights. Run:ai license

A busier GPU can still finish fewer useful jobs

GPU utilization is an appealing measure because idle equipment is expensive. For users waiting on a result, completion time and reliable service also matter. A scheduler can increase activity while changing which jobs finish first. A trial should therefore record queue waits, completion times, interruptions and failed runs alongside hardware activity.

Earlier scheduling research illustrates why those measures differ. In the 2021 Pollux paper, Aurick Qiao and colleagues studied a system that adjusted both resource assignments and training parameters. They used a measure that accounted for computational throughput and the statistical efficiency of training. Their experiments concerned Pollux, not this Run:ai release, but their approach identifies a practical issue: processing more data per second does not always advance training by the same amount. Pollux research

For a Run:ai evaluation, comparing the same mix of jobs before and after a scheduling change would expose who benefits and who waits longer. Results should be separated by workload class. A lower average wait can conceal longer waits for the largest jobs, while higher utilization can accompany more interrupted work. Those are proposed evaluation measures, not outcomes established by NVIDIA’s announcement.

Open-source availability makes that investigation easier to inspect and customize. The next important evidence will be how specific installations behave under their own demand, allocation rules and support arrangements. A team choosing Run:ai now has more control over those decisions, and a clear reason to assign responsibility for each before making the system central to its computing operations.