Description

You will design, build, and maintain scalable lakehouse architectures and end-to-end data pipelines within a Microsoft Fabric environment to support analytics and reporting.

This role is on-site.

Responsibilities

• Develop scripts and tools to ingest, transform, and deliver data using MS Fabric components.
• Design and implement scalable frameworks for batch and real-time data ingestion from APIs, databases, and streaming platforms.
• Create and manage end-to-end ETL/ELT pipelines for data cleansing, enrichment, and transformation.
• Implement data transformation logic using Spark jobs and notebooks, leveraging Delta Lake and Delta Tables.
• Maintain OneLake architecture to ensure reliable data availability for reporting needs.

Required Skills

• 5+ years of experience in data development or engineering roles.
• Proficiency with Microsoft Fabric components: Lakehouse, OneLake, Pipelines, and Notebooks.
• Strong experience with Apache Spark, PySpark, and Spark SQL for large-scale data processing.
• Strong knowledge of SQL and Python programming languages.
• Experience designing and implementing ETL/ELT data pipelines.
• Understanding of Delta Lake and Delta Tables for data management.
• Ability to integrate API and streaming data sources into processing frameworks.

Preferred Skills

• Experience with real-time streaming data architectures.
• Knowledge of advanced data modeling techniques for lakehouse environments.

Key Skills

Education

Any Graduate