Uber’s Journey to Ray on Kubernetes: Ray Setup

Uber enhanced its machine learning platform by migrating workloads to Kubernetes in early 2024. The migration aimed to solve pain points such as manual resource management, inefficient resource utilization, and inflexible capacity planning. The architecture designed included federated resource management featuring a global control plane, cluster management, job execution, monitoring, job routing based on organizational hierarchies, and error handling to ensure a reliable experience for users.