Run:ai

Overview

Run:ai is an MLOps platform designed to facilitate the management and orchestration of AI workloads, particularly those reliant on Graphics Processing Units (GPUs). Established in 2018 and later acquired by NVIDIA, the platform focuses on optimizing the utilization of expensive GPU infrastructure in enterprise environments. It addresses common challenges faced by machine learning teams, such as inefficient resource allocation, long queue times for experiments, and complex infrastructure management.

The system operates as a Kubernetes-native solution, abstracting the underlying infrastructure complexities from ML engineers and researchers. This allows developers to focus on model development and experimentation rather than infrastructure provisioning or cluster management. Run:ai achieves this through capabilities like GPU virtualization, dynamic workload scheduling, and intelligent resource pooling. For instance, a single physical GPU can be shared among multiple users or tasks, dynamically allocating fractional GPU resources based on demand, which can improve overall hardware utilization rates. This approach differs from traditional static allocation methods where GPUs might sit idle while waiting for dedicated tasks.

Target users include large enterprises, research institutions, and organizations with significant investments in AI infrastructure that need to scale their deep learning initiatives efficiently. It is particularly suited for environments where multiple data scientists and ML engineers are concurrently running experiments, training models, and deploying AI applications. The platform provides tools for managing the entire lifecycle of an AI project, from initial experimentation and model training to deployment and monitoring. Its emphasis on a flexible research environment allows teams to iterate more quickly, supporting a range of frameworks such as TensorFlow, PyTorch, and MXNet without requiring changes to existing model code. By providing a unified control plane for AI compute, Run:ai aims to reduce operational overhead and accelerate the time-to-market for AI-driven products and services.

The platform's architecture is built on Kubernetes, enabling seamless integration into existing cloud-native infrastructures. This design choice aligns with the broader industry trend towards containerization and orchestration for scalable application deployment, as noted in various enterprise technology analyses, including those from Thoughtworks Technology Radar on Kubernetes as a Platform. For developers, this means interacting with familiar Kubernetes primitives while benefiting from Run:ai's specialized AI-aware scheduling and resource management capabilities. The web-based dashboard provides administrators and users with visibility into cluster health, resource usage, and job status, facilitating operational oversight and debugging.

Key features

• AI Compute Orchestration: Manages and schedules diverse AI workloads, including training, inference, and data processing, across shared GPU clusters. It allows for dynamic prioritization and resource allocation based on project needs and user roles.
• GPU Virtualization and Pooling: Enables multiple users or jobs to share fractional portions of a single GPU, improving utilization rates and reducing idle resources. This allows for more efficient use of expensive hardware, contrasting with traditional dedicated GPU allocation for each task.
• Dynamic Resource Allocation: Automatically adjusts GPU and CPU resources assigned to jobs based on real-time demand and predefined policies, minimizing queue times and maximizing throughput.
• MLOps Platform Capabilities: Provides tools for managing the end-to-end machine learning lifecycle, including experiment tracking, model versioning, and deployment automation within a Kubernetes environment.
• Kubernetes-Native Integration: Built on Kubernetes, allowing for compatibility with existing containerized workflows and infrastructure management tools. Users interact through standard Kubernetes commands and APIs.
• Centralized Dashboard: Offers a web-based interface for monitoring cluster health, GPU utilization, job status, and resource consumption across all AI projects and users.
• Multi-Framework Support: Compatible with popular deep learning frameworks such as TensorFlow, PyTorch, Keras, and MXNet, allowing data scientists to use their preferred tools without modification.
• Project and User Management: Provides capabilities for creating projects, assigning users, setting quotas, and managing access control to ensure secure and organized collaboration on AI initiatives.

Pricing

Run:ai offers custom enterprise pricing. Specific details are not publicly disclosed and require direct engagement with their sales team to obtain a quote tailored to an organization's specific infrastructure, scale, and feature requirements.

Common integrations

• Kubernetes: As a Kubernetes-native platform, Run:ai integrates directly with any Kubernetes cluster, leveraging its orchestration capabilities for containerized AI workloads (Run:ai installation documentation).
• NVIDIA GPUs: Designed to optimize NVIDIA GPU utilization, integrating with NVIDIA's hardware and software stack for compute-intensive tasks (Run:ai CLI guide).
• Container Registries (e.g., Docker Hub, AWS ECR, Google Container Registry): Connects with standard container registries to pull and manage Docker images for AI workloads.
• Cloud Providers (e.g., AWS, Azure, Google Cloud): Deploys and manages AI workloads on Kubernetes clusters hosted on major cloud platforms.
• Monitoring Tools (e.g., Prometheus, Grafana): Provides metrics and logs that can be integrated with external monitoring and observability tools for deeper insights into cluster performance and job status.
• Source Control Management (e.g., Git): Facilitates MLOps workflows by integrating with Git repositories for code versioning and experiment reproducibility.

Alternatives

• Kubeflow: An open-source machine learning toolkit for Kubernetes, providing components for deploying, managing, and scaling ML workloads.
• Weights & Biases: A platform for experiment tracking, model versioning, and collaboration in machine learning projects, often used in conjunction with underlying compute infrastructure.
• Domino Data Lab: An enterprise MLOps platform that provides an environment for data scientists to develop, deploy, and manage models at scale.

Getting started

To begin using Run:ai, you typically start by deploying the Run:ai control plane onto an existing Kubernetes cluster. Once installed and configured, users can submit AI jobs using the Run:ai CLI or by defining Kubernetes custom resources. The following example demonstrates submitting a simple PyTorch training job using the Run:ai CLI, requesting specific GPU resources.

First, ensure you have the runai CLI installed and configured to connect to your cluster. Then, you can define a job. This example assumes you have a Docker image containing your PyTorch training script.

# 1. Log in to the Run:ai cluster (if not already logged in)
runai login

# 2. Create a new project (if you don't have one)
runai project create my-pytorch-project

# 3. Switch to your project
runai config project my-pytorch-project

# 4. Submit a PyTorch training job requesting 1 GPU and 8GB of memory
# Replace 'my-docker-registry/my-pytorch-image:latest' with your actual image
# Replace 'python train.py --epochs 10' with your training command
runai submit \
  --name pytorch-training-job \
  --gpu 1 \
  --memory 8Gi \
  --image my-docker-registry/my-pytorch-image:latest \
  --command "python train.py --epochs 10"

# 5. Monitor the status of your job
runai list jobs

# 6. View logs for a specific job
runai logs pytorch-training-job

This command instructs Run:ai to schedule a containerized job named pytorch-training-job on the Kubernetes cluster. It requests one GPU and 8 GiB of memory. The job will use the specified Docker image and execute the python train.py --epochs 10 command within the container. Run:ai's scheduler will then find available resources and launch the job, managing its lifecycle and resource consumption. This provides a simplified interface for data scientists to launch complex, resource-intensive AI tasks without needing to write detailed Kubernetes YAML manifests directly.