Description

You will design, build, and orchestrate ETL/ELT pipelines using Azure Databricks, implementing batch data ingestion and transformations with PySpark and Spark SQL. You will architect and maintain Lakehouse and analytical warehouse models using Delta Lake, ensuring data quality, reliability, lineage, and governance. You will collaborate with security and platform teams to enforce data access controls and manage CI/CD pipelines using Git.

This role is on-site.

Responsibilities

• Design and orchestrate ETL/ELT pipelines using Azure Databricks.
• Implement batch data ingestion and transformations using PySpark and Spark SQL.
• Architect Lakehouse and analytical warehouse models leveraging Delta Lake.
• Ensure data quality, reliability, lineage, and governance across the platform.
• Enforce data access controls and best practices with security teams.

Required Skills

• 6-10 years of hands-on experience in data engineering.
• Strong, proven expertise in Azure Databricks and Delta Lake.
• Advanced proficiency in SQL and distributed data processing concepts.
• Experience leading, mentoring, or supervising data engineering teams.
• Strong experience performing code reviews and solution design validations.
• Knowledge of data security and access management (RBAC).
• Experience with CI/CD pipelines using Git.

Preferred Skills

• Experience with Azure Synapse or Fabric.
• Experience with Azure Data Factory.

Key Skills

Education

Bachelor's degree