Description

You will build and scale the infrastructure required to serve machine learning models at scale.

Responsibilities

• Design and implement scalable model serving platforms for batch and real-time inference.
• Build model deployment pipelines featuring automated testing and validation.
• Develop monitoring, logging, and alerting systems for ML services.
• Create infrastructure for A/B testing and model experimentation.
• Implement model versioning, rollback capabilities, and efficient scaling strategies for ML workloads.

Required Skills

• 7+ years of software engineering experience, with 3+ years focused on ML serving or infrastructure.
• Strong expertise in Kubernetes and container orchestration.
• Proficiency in Python and experience with high-performance serving.
• Experience with model serving technologies including TensorFlow Serving, Triton, or KServe.
• Deep knowledge of distributed systems and microservices architecture.
• Hands-on experience with CI/CD pipelines and GitOps workflows.
• Strong background in monitoring and observability tools.
• Any Graduate degree.

Preferred Skills

• Experience with TorchServe, BentoML, or multi-framework support via Triton.
• Expertise in model runtime optimizations such as quantization (INT8, FP16), pruning, or kernel optimizations.
• Knowledge of GPU infrastructure management and low-latency serving architectures.

Key Skills

Education

Any Graduate