Description

Build and maintain large-scale data pipelines, real-time streaming solutions, and cloud infrastructure on AWS.

This role is on-site.

Responsibilities

• Build and maintain large-scale data processing pipelines using Apache Spark.
• Develop real-time data streaming solutions with Kafka.
• Orchestrate complex workflows using Apache Airflow.
• Deploy and manage data infrastructure on AWS, specifically leveraging AWS EMR.
• Write efficient, scalable code in Python and SQL.

Required Skills

• 5+ years of hands-on Data Engineering experience.
• Strong proficiency in Python and SQL.
• Experience with Apache Spark for distributed data processing.
• Hands-on experience with Kafka for real-time streaming.
• Familiarity with Apache Airflow for workflow orchestration.
• Experience with AWS EMR (Elastic MapReduce).
• Cloud platform experience, with AWS preferred.

Key Skills

Education

Any Graduate