Description

Key Responsibilities

• Design, develop, and optimize data pipelines handling terabyte-scale datasets
• Work with complex algorithms to process and analyze large volumes of data efficiently
• Optimize code performance for scalability and high-throughput systems
• Build and maintain containerized, serverless data platforms using Kubernetes
• Support migration efforts from EMR/EC2-based systems to Kubernetes-based architecture
• Maintain and enhance existing Kubernetes infrastructure
• Develop and manage CI/CD pipelines using tools like GitLab and Bitbucket
• Collaborate on requirements documentation, system design, and implementation
• Contribute to emerging initiatives involving:
• GenAI integration
• AI agents and automation frameworks
• Technologies such as Kiro (Amazon GenAI) and MCP (Model Context Protocol)

Technical Environment

• Cloud Platforms: AWS (current), with exposure to Google Cloud and open-source ecosystems
• Core Technologies:
• Kubernetes (primary focus)
• Elasticsearch
• Big Data frameworks (EMR, distributed systems)
• Programming Languages: Python, SQL, Scala (flexible)
• DevOps & Tooling: GitLab, Bitbucket, CI/CD pipelines

Required Skills & Experience

• Strong experience with Kubernetes, including deployment, migration, and infrastructure management
• Experience working with large-scale data (terabytes) and distributed systems
• Proficiency in at least one: Python, SQL, or Scala
• Understanding of data processing optimization techniques
• Familiarity with cloud-native architectures and containerization
• Experience with CI/CD pipelines and modern DevOps practices
• Exposure to AI/GenAI tools, agents, or related frameworks is a strong plus
• AWS and/or Kubernetes certifications preferred

Key Skills

Education

Any Gradute