Description

Design and implement scalable data pipelines using Azure Databricks and PySpark.

This role is hybrid.

Responsibilities

• Design and implement scalable data pipelines using Azure Databricks (ADB) and PySpark.
• Write efficient and optimized SQL queries to process and analyze large datasets.
• Collaborate with cross-functional teams for data integration, transformation, and validation.
• Optimize and troubleshoot data workflows for performance improvements.
• Ensure best practices in coding, testing, and deployment of data engineering solutions.

Required Skills

• 6+ years of professional experience.
• Expertise in Azure Databricks.
• Strong proficiency in PySpark.
• Advanced SQL querying and optimization skills.
• Experience designing and implementing data pipelines.
• Ability to translate stakeholder requirements into data solutions.

Preferred Skills

• Experience with data validation techniques.
• Knowledge of data integration patterns.

Key Skills

Education

Any Graduate