Description

Design and implement scalable data platforms using Azure Databricks and Lakehouse Architecture.

This role is on-site.

Responsibilities

• Build and maintain batch and real-time streaming pipelines for ingestion and analytics.
• Migrate legacy data platforms to Lakehouse Architecture using Delta Lake and MLflow.
• Define and enforce data governance, security, and quality standards.
• Mentor data engineers and establish Databricks engineering best practices.
• Integrate Databricks with Azure Data Factory, Kafka, and BI tools.

Required Skills

• Strong expertise in Azure Databricks and Microsoft Azure.
• Proficiency in SQL, Python, Scala, and Apache Spark.
• Hands-on experience with CI/CD, DevOps, and orchestration tools like Apache Airflow or Azure Data Factory.
• Understanding of distributed computing, performance optimization, and data security.
• Bachelor's or Master's degree in Computer Science, Data Science, or a related field.
• 8+ years of experience in data architecture or engineering.

Preferred Skills

• Experience with Data Mesh, Data Fabric, or Enterprise Data Architecture.
• Exposure to ERP platforms such as SAP or Salesforce.

Key Skills

Education

Bachelor's or Master's degrees