Description

You will own the design, build, and maintenance of ELT/ETL pipelines using Microsoft Fabric. You transform, cleanse, and enrich data for reporting and analysis. You optimize storage and retrieval for large datasets. You implement automation to reduce manual intervention in data processing workflows. You collaborate with Platform Architects and Report Developers to ensure infrastructure and data formats meet business needs.

Responsibilities

• Build and maintain ELT/ETL pipelines using Microsoft Fabric tools for efficient data ingestion.
• Apply transformations and cleansing rules to ensure data is ready for analysis and reporting.
• Optimize storage and retrieval performance for large datasets using OneLake and Delta Lake formats.
• Implement automation for data processing and integration workflows to reduce manual intervention.
• Partner with Platform Architects and Report Developers to align infrastructure and data usability.

Required Skills

• 8+ years of data engineering experience.
• Microsoft Fabric (Data Factory, Lakehouse, Warehouse).
• PySpark, Spark SQL, and Scala for complex data processing.
• Experience with Spark Job Definitions and Notebook orchestration.
• OneLake and ADLS Gen2 storage structures, including Delta tables and partitioning strategies.
• Monitoring Hubs for pipeline and job performance tracking.
• Bachelor’s degree in Computer Science, Engineering, or a relevant field.

Preferred Skills

• Azure or AWS Cloud Certifications.
• Experience in the Manufacturing domain.

Key Skills

Education

Any Graduate