Description

Design and lead scalable data architectures on the Azure platform, ensuring performance and integrity.

Responsibilities

• Design scalable data architectures using Databricks, Unity Catalog, Azure Data Lakehouse, and Azure Data Factory.
• Lead and mentor data engineers, full stack developers, and QA teams throughout the project lifecycle.
• Oversee data engineering projects from extraction to end-user consumption, ensuring quality and security.
• Collaborate with data scientists, analysts, and business stakeholders to define technical requirements.
• Implement specific use cases for Unity Catalog while maintaining data quality and integrity.

Required Skills

• 10+ years of experience in data engineering.
• Extensive experience with Databricks, Unity Catalog, Azure Data Lakehouse, and Azure Data Factory.
• Proficiency in Python, PySpark, SQL, and Scala.
• Deep understanding of Medallion Architecture, data modeling, and ETL/ELT processes.
• Experience with Microsoft Azure services for performance and scalability.
• Familiarity with big data technologies including Hadoop, Spark, and Kafka.
• Experience with DevOps practices and CI/CD tools.
• Knowledge of full stack web development technologies such as React and .Net.

Preferred Skills

• Bachelor's or Master's degree in Computer Science, IT, or a related field.

Key Skills

Education

Bachelor's degree in Computer Science