Description

Design, build, and optimize scalable data architectures and integration workflows for large-scale datasets.

This role is hybrid.

Responsibilities

• Develop and maintain data integration workflows using Azure Data Factory.
• Process large datasets efficiently using PySpark to ensure data quality and performance.
• Monitor and optimize data pipelines for reliability, efficiency, and scalability.
• Implement data security and governance policies across all data pipelines.
• Translate business requirements into technical data solutions through stakeholder engagement.

Required Skills

• 9 to 13 years of hands-on data engineering experience.
• Expertise in Azure Data Factory for ETL and data integration.
• Proficiency in Databricks for data analytics and pipeline creation.
• Advanced skills in PySpark for large-scale data processing.
• Proven track record designing scalable data solutions.
• Knowledge of data governance and security best practices.
• Strong problem-solving skills and attention to detail.
• Effective communication and documentation abilities.

Preferred Skills

• Experience with Azure Synapse.
• Experience with Azure Data Lake.

Key Skills

Education

Any Gradute