Description

Build and maintain scalable data pipelines and ETL workflows using Databricks and PySpark.

This role is hybrid.

Responsibilities

• Develop and optimize PySpark jobs on Databricks to process large-scale datasets.
• Build and maintain ETL/ELT workflows and end-to-end data pipelines for downstream analytics.
• Implement data transformations and modeling using dbt (Data Build Tool) with version-controlled SQL models.
• Ensure data reliability through testing, monitoring, and data quality checks across pipeline stages.
• Collaborate with analytics and engineering teams to integrate data sources and deliver curated datasets.

Required Skills

• 5-12 years of relevant experience in data engineering.
• Strong proficiency in Databricks and PySpark for scalable batch data processing.
• Hands-on experience with ETL and ELT development for production data workflows.
• Proficiency in Python.
• Experience with dbt (Data Build Tool) for SQL-based transformations, modeling, and testing.
• Strong understanding of pipeline design and reliability.

Preferred Skills

• Experience with additional orchestration, monitoring, and data quality tooling.

Key Skills

Education

Any Graduate