Description

You will manage and operate large-scale, GPU-accelerated AI platforms and LLM inference systems.

This role is on-site.

Responsibilities

• Deploy and manage LLM inference systems on Kubernetes, focusing on production-grade LLM pipelines.
• Design and operate containerized microservices for deploying models in production environments.
• Lead end-to-end LLMOps processes, covering model versioning, engine builds, and automated rollouts.
• Develop observability for inference systems, tracking GPU health, latency, and throughput via telemetry.
• Operate and support mission-critical systems, including incident and change management processes.

Required Skills

• 10+ years of experience deploying, managing, and troubleshooting containerized services at scale on Kubernetes (OpenShift).
• Expertise configuring and tuning LLMs using TensorRT-LLM and Triton Inference Server.
• Experience managing ML/LLMOps pipelines for deploying inference services in production.
• Proven ability to optimize models using techniques like mixed precision, quantization, sharding, and batching.
• Experience with API Design and building microservices.
• Familiarity with standard operational processes for mission-critical systems.
• Experience with GPU-aware scheduling and load balancing across multi-node clusters.

Preferred Skills

• Any Graduate education.

Key Skills

Education

Any Graduate