Description

What's the Job?

• Develop scripts, processes, and tools to ingest, transform, and deliver data within a MS Fabric environment to support analytics and reporting needs.
• Design, build, and maintain scalable lakehouse architecture using Microsoft Fabric and OneLake.
• Create and manage end-to-end data pipelines for ingestion, transformation, and serving of data.
• Develop scalable frameworks for batch and real-time data ingestion from sources such as APIs, databases, and streaming platforms.
• Implement data cleansing, enrichment, and transformation logic using Spark jobs and notebooks.

What's Needed?

• Proficiency with Microsoft Fabric components including Lakehouse, OneLake, Pipelines, and Notebooks.
• Experience with Apache Spark, PySpark, and Spark SQL for large-scale data processing.
• Strong knowledge of SQL and Python programming languages.
• Understanding of Delta Lake and Delta Tables for data management.
• Experience designing and implementing data pipelines (ETL/ELT) and integrating API and streaming data sources

Key Skills

Education

Any Graduate