Description

You will operate, optimize, and automate large-scale data infrastructure across AWS and on-premises Hadoop platforms. This role focuses on platform stability, performance, observability, and incident response for mission-critical data services.

This role is remote.

Responsibilities

• Maintain highly available, scalable, and secure data infrastructure on AWS and Cloudera Hadoop environments.
• Drive operational excellence by automating repetitive tasks and reducing incident frequency through Infrastructure as Code.
• Monitor platform health, troubleshoot complex distributed systems, and lead root cause analysis to improve system resiliency.
• Collaborate with engineering and data teams to support deployments, integrations, and performance optimization.
• Participate in on-call rotations and enhance observability and alerting processes to reduce operational noise.

Required Skills

• 10+ years of experience in Site Reliability Engineering with a focus on availability and performance.
• Deep hands-on expertise with AWS services: EMR, EKS, MSK, Athena, Glue, IAM, S3, and VPC networking.
• Extensive operational experience managing Hadoop clusters, including on-premises Cloudera CDH/CDP and Kerberos security.
• Strong proficiency in Python for automation and Terraform for Infrastructure as Code.
• Hands-on experience with Apache Spark, Apache Iceberg, and big data platform architecture.
• Proficiency in monitoring and observability tools such as AWS CloudWatch, Datadog, and PagerDuty.
• Strong Linux administration skills, including JVM tuning, thread/heap dump analysis, and log troubleshooting.
• Experience with Agentic AI concepts applied to infrastructure automation.

Preferred Skills

• Proven track record of delivering services within defined SLAs and minimizing production downtime.
• Experience supporting 24x7 operations across US and India time zones.

Key Skills

Education

Any Graduate