Description

You will engineer and optimize data pipelines using Azure services, specifically focusing on Subrogation solutions.

This role is hybrid.

Responsibilities

• Develop and refactor legacy Cursor implementation code into PySpark solutions leveraging distributed computing.
• Design and implement data persistence models on ADLS, optimizing Read and Write performance.
• Conduct performance testing and optimization of data processes alongside architects.
• Build and maintain data solutions using Databricks, PySpark, Python, and ADF.

Required Skills

• 5+ years of professional experience in data engineering.
• Strong proficiency with Databricks and PySpark.
• Hands-on experience with Azure Data Factory (ADF).
• Proficiency in Python with a command of object-oriented and functional programming models.
• Experience optimizing data persistence layers on ADLS for I/O efficiency.
• Understanding of distributed computing paradigms.

Preferred Skills

• Experience specifically within the Subrogation domain.

Key Skills

Education

Any Graduate