Description

Key Responsibilities

• Design, deploy, and maintain production-grade Amazon EKS clusters optimized for petabyte-scale data processing, ensuring high availability, resiliency, and fault tolerance.
• Manage and operate Kubernetes environments within secure, air-gapped private VPCs, including administration of private container registries and package repositories.
• Troubleshoot and resolve complex distributed systems issues across EKS, Karpenter, and Apache Spark, including scheduling bottlenecks, node provisioning delays, and large-scale workload failures.
• Implement and optimize Karpenter consolidation and disruption policies to balance infrastructure efficiency, cost management, and workload stability.
• Develop and manage Spot and On-Demand instance strategies, including robust interruption handling and workload recovery mechanisms.
• Define and enforce ResourceQuotas, LimitRanges, and PriorityClasses to ensure efficient resource allocation and prevent resource contention.
• Configure and manage persistent storage solutions using Amazon EBS CSI Driver and Amazon EFS CSI Driver to support high-performance data processing workloads.
• Optimize storage architecture for throughput, scalability, durability, and application resiliency.
• Design and implement proactive monitoring, logging, alerting, and anomaly detection solutions to identify and remediate system issues before they impact production operations.
• Architect fault-tolerant solutions utilizing checkpointing, retry mechanisms, and graceful degradation strategies to minimize downtime and recomputation.
• Develop, maintain, and manage Kubernetes configurations and operational processes within highly secured, internet-restricted environments.

Required Qualifications

• Extensive hands-on experience designing, deploying, and supporting Kubernetes platforms in enterprise production environments.
• Deep expertise with Amazon EKS and cloud-native technologies.
• Strong experience with Kubernetes cluster autoscaling solutions, including Karpenter.
• Proven experience supporting large-scale Apache Spark workloads and distributed systems.
• Strong troubleshooting and performance tuning skills across Kubernetes, AWS infrastructure, networking, and storage.
• Experience working within highly secure, restricted-access, or air-gapped environments.
• Strong understanding of Kubernetes resource management, scheduling, observability, and production operations.

Preferred Qualifications

• Kubernetes certifications such as CKA, CKAD, or CKS.
• AWS certifications.
• Experience contributing to open-source Kubernetes, Karpenter, or Apache Spark projects.
• Knowledge of FinOps methodologies and cloud cost optimization at scale.
• Background supporting data engineering, analytics, or large-scale data platform environments

Key Skills

Education

Any Graduate