Description

Design and develop end-to-end data pipelines on Azure using ADF, Databricks, and Synapse. You will own the implementation of distributed data workflows, CI/CD processes, and performance tuning for cloud data solutions.

This role is on-site.

Responsibilities

• Build end-to-end data pipelines using Azure Data Factory and Azure Databricks.
• Develop distributed data workflows and processing solutions in Python, PySpark, Scala, and Hive.
• Implement CI/CD pipelines for data applications to ensure reliable deployment.
• Collaborate with teams to gather requirements and deliver high-quality data architecture.
• Ensure data quality, performance tuning, and adherence to cloud best practices.

Required Skills

• 5+ years of experience in data engineering.
• Strong proficiency in Python and PySpark.
• Hands-on experience with Azure Data Factory (ADF) and Azure Databricks (ADB).
• Experience with Scala and Hive for data processing.
• Knowledge of Azure SQL Database, SQL Data Warehouse, and Synapse Analytics.
• Proven ability to implement CI/CD pipelines.
• Bachelor's degree in a relevant field.

Key Skills

Education

Bachelor's degree