Description

You will build and automate large-scale data processing systems for batch and streaming workloads.

Responsibilities

• Build scalable Spark data pipelines using Airflow scheduler and executor frameworks.
• Deploy production-quality ETL pipelines using established frameworks.
• Analyze large datasets to identify gaps, inconsistencies, and provide actionable insights.
• Automate high-performance data processing systems to drive business growth.

Required Skills

• 8+ years of relevant industry experience.
• Expertise in building batch data pipelines using Spark and Scala.
• Strong experience with ETL frameworks such as Airflow, Flume, or Oozie.
• Proficiency in SQL and relational database query authoring.
• Deep understanding of distributed storage and compute, specifically S3, Hive, and Spark.
• Software engineering proficiency in Java or Python.
• Experience with version control using GitHub.
• Degree in any field of study.

Preferred Skills

• Ability to communicate technical insights clearly in written and verbal formats.

Key Skills

Education

ANY GRADUATE