Description

You will design and develop scalable data pipelines on Azure, ensuring data quality and performance across production environments.

This role is on-site.

Responsibilities

• Build and maintain ETL/ELT workflows using PySpark and Python for large datasets.
• Design data models and manage database solutions to support business requirements.
• Optimize data processing jobs and troubleshoot issues in production pipelines.
• Implement data integration between multiple systems and sources.
• Maintain documentation for data workflows and collaborate with engineering teams.

Required Skills

• 6-10 years of overall IT experience with at least 5 years in Azure Data Engineering.
• Strong proficiency in Python and PySpark for data processing and transformation.
• Expertise in SQL for querying, transformation, and performance tuning.
• Experience building and maintaining data pipelines in Azure.
• Knowledge of database technologies and big data processing concepts.
• Strong debugging and performance optimization skills.

Preferred Skills

• Hands-on experience with AWS data services.
• Experience working in distributed systems and large-scale data environments.

Key Skills

Education

Any Graduate