Description

You will design and build scalable data solutions on Azure, focusing on Databricks, PySpark, and Python.

This role is on-site.

Responsibilities

• Develop and maintain data pipelines using Azure Data Factory and Databricks.
• Implement data validation processes to ensure quality and integrity.
• Build AI/ML models and integrate them into data workflows.
• Collaborate within an Agile team to deliver robust data engineering solutions.
• Optimize PySpark and Python code for performance at scale.

Required Skills

• 9+ years of total software engineering experience.
• 3-5 years of hands-on experience with Databricks and PySpark.
• 3+ years of experience with Azure Data Factory.
• Strong proficiency in Python for data engineering tasks.
• Practical experience with AI/ML model development and deployment.
• Experience implementing data validation frameworks.
• Bachelor's or Master's degree in Engineering or related field.

Preferred Skills

• Experience with Azure cloud services beyond Data Factory.
• Strong background in Agile development methodologies.

Key Skills

Education

Bachelor's or Master's degrees