Job Description

Responsibilities

• Design and build orchestration capabilities for Kubernetes Operators, container runtimes, and lifecycle management of jobs, services, and stateful workloads.
• Develop multi-tenant resource and quota systems supporting priorities, preemption, fair sharing, elasticity, cross-cluster scheduling, resource pooling, GPU utilization, and FinOps.
• Build online model-serving lifecycle orchestration for model and image distribution, deployment, upgrades, rollback, autoscaling, multi-cluster operation, and disaster recovery.
• Develop serving orchestration and traffic management for disaggregated serving clusters, including topology-aware scheduling, KV Cache affinity, intelligent request routing, and QoS/SLA management.

Requirements

• Completing or recently completed a PhD in Computer Science, Software Engineering, Artificial Intelligence, or a related technical field.
• Proficiency in at least one of Go, C++, or Python, with a foundation in data structures, algorithms, and software engineering principles.
• Familiarity with Linux, operating systems, computer networks, concurrent programming, and distributed systems.
• Ability to investigate systems using source code, metrics, logs, profiling, and experiments.
• A systematic and quantitative approach to defining measurements, testing hypotheses, and validating system improvements.
• Demonstrated ownership and collaboration through coursework, research, internships, open-source contributions, or other engineering projects.
• Preferred experience includes Kubernetes, container runtimes, resource scheduling, quota management, multi-tenant systems, or FinOps.
• Preferred experience includes contributions to Kubernetes, Volcano, Koordinator, or OpenKruise infrastructure projects.
• Preferred experience includes vLLM, SGLang, Triton, KServe, Ray Serve, KV Cache, Continuous Batching, Prefill/Decode disaggregation, or model parallelism.
• Preferred experience includes online services, gateways, traffic management, autoscaling, performance optimization, highly available distributed systems, GPU/NPU programming, heterogeneous resource scheduling, model distribution, or inference performance analysis.

Benefits

• Candidates must be able to commit to an onboarding date by the end of the year and should state their availability and graduation date clearly in their resume.

Job Tags

Similar Jobs