Description

You will design and build the inference service platform that serves as the backbone for AI models at scale.

This role is on-site.

Responsibilities

• Architect core platform components including the AI Gateway, Resource Orchestrator, Runtime Engines, and Autoscaler systems.
• Design modular, scalable, and extensible low-level architectures for inference infrastructure.
• Build deployment pipelines for seamless model updates with zero-downtime deployments and intelligent routing.
• Implement high-performance software for low-latency, high-throughput inference serving in distributed systems.
• Build telemetry and observability stacks for inference metrics, performance tracking, and debugging.

Required Skills

• 5+ years of experience building and scaling backend systems, distributed platforms, or inference infrastructure.
• Strong understanding of AI/ML inference systems.
• Proficiency in Golang, Python, and microservices architecture.
• Experience with Kubernetes, Docker, and containerized deployments.
• Familiarity with Redis, PostgreSQL, and Apache Kafka.
• Knowledge of load balancing and API gateway patterns.

Preferred Skills

• Experience with inference engines like vLLM, TGI, or TensorRT-LLM.
• Familiarity with distributed systems architecture and autoscaling strategies.

Key Skills

Education

Any Graduate