Description

You will design, build, and optimize scalable data architectures and integration workflows.

Responsibilities

• Develop data integration workflows using Azure Data Factory.
• Process large datasets efficiently using PySpark to ensure data quality.
• Monitor and optimize data pipelines for performance, reliability, and efficiency.
• Provide technical guidance to team members and document data processes.
• Translate business requirements into technical data solutions through stakeholder engagement.
• Implement data security and governance policies across all pipelines.

Required Skills

• 9 to 13 years of hands-on data engineering experience.
• Expertise in Azure Data Factory for ETL and data integration.
• Proficiency in Databricks for data analytics and pipeline creation.
• Advanced skills in PySpark for large-scale data processing.
• Proven track record designing scalable data solutions.
• Knowledge of data governance and security best practices.
• Strong problem-solving skills and attention to detail.
• Effective communication and documentation abilities.

Preferred Skills

• Experience with Azure Synapse.
• Experience with Azure Data Lake.

Key Skills

Education

Any Gradute