Description

You will own the development and optimization of large-scale data pipelines using PySpark and Databricks.

This role is hybrid.

Responsibilities

• Develop and optimize PySpark jobs on Databricks for scalable batch data processing.
• Build and maintain ETL/ELT workflows and end-to-end data pipelines for downstream analytics.
• Implement data transformations and modeling using dbt with version-controlled SQL models.
• Ensure data reliability through testing, monitoring, and data quality checks across pipeline stages.
• Collaborate with analytics and engineering teams to integrate data sources and deliver curated datasets.

Required Skills

• 5-12 years of relevant experience in data engineering.
• Strong proficiency in Databricks and PySpark for scalable batch processing.
• Hands-on experience with ETL and ELT development for production data workflows.
• Expertise in dbt (Data Build Tool) for SQL-based transformations, modeling, and testing.
• Strong understanding of pipeline design, reliability, and implementation.
• Proficiency in Python.

Preferred Skills

• Experience with additional orchestration, monitoring, and data quality tooling.

Key Skills

Education

Any Graduate