Description

You will develop and optimize ETL pipelines using PySpark and Databricks to support data migration projects.

This role is remote.

Responsibilities

• Develop and optimize ETL pipelines using PySpark and Databricks.
• Migrate legacy on-premise applications from Informatica to cloud-based solutions.
• Cleanse and transform unstructured and structured data from multiple vendors, including Bloomberg.
• Implement data rules based on business requirements to create accurate data models.
• Collaborate with team members in an Agile environment.

Required Skills

• 5+ years of experience in development, data modeling, or related fields.
• Proficiency in PySpark with a focus on data processing and pipeline development.
• Experience with Databricks workflows and cloud-based data environments.
• Strong understanding of data cleansing, hierarchy, and applying business rules.
• Ability to work effectively in a small, collaborative team.
• Experience migrating from Informatica.

Preferred Skills

• Experience with Bloomberg data sources.
• Background in legacy application modernization.

Key Skills

Education

Any Graduate