Description

Key Responsibilities

• Design and develop scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Python.
• Implement Delta Lake, Delta Live Tables (DLT), and Unity Catalog.
• Build and optimize data pipelines using Azure Data Factory and ADLS Gen2.
• Implement CI/CD pipelines using Azure DevOps and Git.
• Optimize data processing performance, scalability, and cost.
• Ensure data quality, governance, security, and operational excellence.
• Mentor developers and collaborate with cross-functional teams.

Mandatory Skills

• Azure Databricks, PySpark, Apache Spark, Python, and SQL
• Azure Data Factory (ADF) and ADLS Gen2
• Delta Lake, Delta Live Tables (DLT), and Unity Catalog
• Databricks Workflows
• CI/CD using Azure DevOps and Git
• Data Engineering, ETL/ELT, and Data Warehousing

Key Skills

Education

Bachelor's degree