Description

You will lead the design and implementation of scalable, secure data pipelines and data modeling strategies.

Responsibilities

• Translate business requirements into low-level technical designs.
• Design and implement fault-tolerant data pipelines using PySpark and MS Fabric notebooks.
• Mentor junior engineers and perform peer code reviews.
• Drive automation in data ingestion, movement, and access workflows.
• Lead data migration projects from legacy warehouses to Lakehouse architectures using Delta Lake.

Required Skills

• 8 to 12 years of experience in data engineering.
• At least 1 year of hands-on experience with MS Fabric and Azure cloud.
• Proficiency in Python, PySpark, and SQL for data manipulation and transformation.
• Experience with Databricks, Azure Synapse, and Azure Data Lake Storage.
• Expertise in ETL processes, data modeling, and creating fact and dimension tables.
• Experience setting up pipelines using Azure Data Factory (ADF) and Notebooks.
• Knowledge of orchestration tools such as Apache Airflow or Azure Data Factory.
• Familiarity with batch processing, streaming, auto-loader, and change data capture (CDC).
• Experience implementing monitoring and alert notifications for data workflows.

Preferred Skills

• DP-700 certification.

Key Skills

Education

Any Graduate