Description

You own the design, development, and maintenance of data pipelines and ETL/ELT processes.

Responsibilities

• Design, develop, and maintain data pipelines and ETL/ELT processes using Microsoft Fabric and PySpark.
• Build and manage Lakehouse and Data Warehouse solutions within the Microsoft Fabric ecosystem.
• Develop scalable data processing workflows using Python and PySpark.
• Write optimized SQL queries for data transformation, analysis, and performance tuning.
• Integrate data from various sources such as APIs, databases, cloud storage, and streaming platforms.

Required Skills

• 8+ years of experience in data engineering.
• Strong hands-on experience with Microsoft Fabric (Lakehouse, Data Factory, Data Warehouse, Notebooks).
• Expertise in PySpark and Spark-based data processing.
• Strong programming skills in Python.
• Advanced knowledge of SQL and database optimization.
• Experience with ETL/ELT pipeline development.
• Understanding of data modeling concepts and data warehousing principles.
• Experience working with large-scale structured and unstructured data.

Preferred Skills

• Experience with Azure Data Factory / Azure Synapse / Databricks.
• Knowledge of Power BI integration with Microsoft Fabric.
• Familiarity with CI/CD pipelines and DevOps practices.

Key Skills

Education

Bachelor's or Master's degrees