Description

8-10 Yr of Experience in Building data architecture using various data engineering techniques

-In-depth knowledge and hands-on experience with Microsoft Fabric components, including Lakehouse, OneLake, Delta Lake format, Fabric Pipelines, Dataflows Gen2, and Spark noteb Data Engineering

-Strong understanding of data lake concepts, including optimal use of OneLake for data storage, Delta Lake -Experience in designing and implementing metadata-driven solutions for data ingestion

-Expertise in various data ingestion patterns, including batch processing (specifically for CSV files)

-Proficient in data modeling standards, potentially including Medallion architecture (Bronze, Silver, Gold layers) -Strong proficiency in PySpark for data processing and transformation within Fabric Spark environments - Advanced SQL skills for querying data, managing database interactions

-Strong analytical and problem-solving skills to assess data requirements, troubleshoot issues

Good to Have:

-Proficiency in PySpark for data processing and transformation within Fabric Spark environments

-Understanding of data governance principles, data security measures

-Knowledge of monitoring data pipeline performance, troubleshooting

Key Skills

Education

Any Graduate