Description

You will build and manage scalable data infrastructure to support feature extraction and ETL processes.

Responsibilities

• Design and implement scalable data pipelines for feature extraction, transformation, and loading using PySpark and Scala.
• Govern and optimize pipelines to ensure reliability, efficiency, and data quality across on-premise and cloud environments.
• Translate data requirements from data scientists, ML engineers, and business stakeholders into technical solutions.
• Implement best practices for data governance, metadata management, and regulatory compliance.
• Lead a team of data engineers through technical guidance and mentorship.

Required Skills

• 6+ years of experience in data engineering.
• Proficiency with PySpark.
• Strong experience with Scala.
• Hands-on experience with Big Data frameworks.
• Experience managing data pipelines in both on-premise and cloud environments.
• Ability to lead and mentor other engineers.

Preferred Skills

• Any Graduate degree.

Key Skills

Education

Any Graduate