Description

Design, develop, and maintain scalable data engineering solutions using Python, Azure Data Factory (ADF), and Azure Databricks.

This role is on-site.

Responsibilities

• Build, orchestrate, and optimize ADF pipelines for reliable data ingestion and transformation workflows.
• Develop and optimize PySpark jobs in Databricks for large-scale, distributed data processing.
• Translate business requirements into efficient cloud-based data integration and analytics solutions.
• Ensure data quality, performance, and reliability across end-to-end ETL/ELT pipelines.
• Monitor, troubleshoot, and resolve failures related to ADF pipelines and Databricks workloads.

Required Skills

• 5–7 years of hands-on experience in data engineering.
• Strong proficiency in Python for data processing.
• Expertise in Azure Data Factory (ADF) pipeline design and optimization.
• Experience with Azure Databricks and PySpark for distributed computing.
• Deep understanding of ETL/ELT patterns and data integration.
• Knowledge of Azure security, governance, and compliance standards.
• Bachelor’s degree in a related field.

Preferred Skills

• Experience working with overseas contractor delivery models.
• Proven track record of delivering scalable, maintainable data platforms.

Key Skills

Education

Bachelor’s degree