Description

You will manage scalable AWS infrastructure and Spark-based data pipelines.

Responsibilities

• Design, implement, and manage secure AWS infrastructure for big data processing.
• Develop and maintain Spark-based data processing pipelines using Scala.
• Implement DevOps practices including CI/CD and monitoring.
• Collaborate with cross-functional teams to deploy and operate big data applications.
• Troubleshoot and resolve issues related to Spark, Scala, and AWS.

Required Skills

• 5 to 10 years of experience as a DevOps Engineer.
• Expertise in AWS infrastructure including EC2, S3, EMR, RDS, Glue, and Athena.
• Strong experience with Spark Core, Spark SQL, and Spark Streaming.
• Proficiency in Scala libraries and frameworks.
• Experience with Jenkins, Git, and Docker.
• Background in managing big data processing workloads.

Key Skills

Education

Any Gradute