Description

Design and manage scalable data architectures and AI/ML integrations on Azure, including building, managing, and optimizing data pipelines using Azure Databricks.

This role is on-site.

Responsibilities

• Build and optimize data pipelines and solutions using Azure Databricks, Synapse, and Purview.
• Design scalable, loosely coupled microservices and API architectures.
• Implement data governance and access control using Unity Catalog.
• Develop ETL/ELT workflows and data modeling structures.
• Integrate data management and AI/ML components into architectural designs.

Required Skills

• 15+ years of experience in data engineering or a related field.
• Extensive experience with Azure Databricks and Azure native services.
• Proficiency in PySpark and Python for large-scale data processing.
• Strong expertise in SQL and Unity Catalog.
• Hands-on experience with Power BI.
• Deep understanding of data modeling concepts and ETL/ELT workflows.
• Experience in microservices and API design.

Preferred Skills

• Experience integrating AI/ML components into data architectures.

Key Skills

Education

Any Graduate