Description

Design, build, and maintain scalable data pipelines and lakehouse architectures on Microsoft Azure.

This role is on-site.

Responsibilities

• Develop ETL/ELT workflows using Azure Data Factory and Databricks to process large volumes of structured and unstructured data.
• Implement data modeling techniques (star/snowflake schema) and dimensional modeling for enterprise analytics.
• Build and optimize big data solutions using PySpark, Scala, and SQL within Azure Databricks.
• Integrate data ingestion frameworks from diverse sources including APIs, databases, and streaming platforms.
• Automate deployment and orchestration using CI/CD pipelines with Azure DevOps or Jenkins.

Required Skills

• 10+ years of experience in data engineering.
• Expertise in Azure Data Factory (ADF), Azure Databricks, and Azure Synapse Analytics.
• Proficiency in Python, SQL, PySpark, and Scala.
• Strong background in Azure Data Lake Storage (ADLS Gen2) and Azure SQL Database.
• Experience with Azure Event Hub, Stream Analytics, or Apache Kafka for streaming data.
• Knowledge of data governance, security (AAD, RBAC), and compliance.
• Familiarity with Power BI or Tableau for data visualization integration.

Preferred Skills

• Experience with Docker and Kubernetes for containerization.
• Background in mentoring junior engineers and establishing best practices.

Key Skills

Education

Bachelor's degree