Description

Build and optimize ETL pipelines using PySpark and Databricks to support data migration and modeling projects.

This role is remote.

Responsibilities

• Develop and optimize ETL pipelines using PySpark and Databricks.
• Migrate legacy on-premise applications from Informatica to cloud-based solutions.
• Cleanse and transform unstructured and structured data from multiple vendors, including Bloomberg.
• Implement data rules based on business requirements to create accurate data models.
• Collaborate with team members in an Agile environment to accelerate development.

Required Skills

• 5+ years of professional experience in data development.
• Proficiency in PySpark for data processing and pipeline development.
• Experience with Databricks workflows and cloud-based data environments.
• Strong understanding of data cleansing, hierarchy, and applying business rules.
• Background in development, data modeling, or related fields.
• Ability to work effectively in a small, collaborative team.
• Any Graduate degree.

Preferred Skills

• Experience with Informatica legacy systems.
• Work with data sources such as Bloomberg.

Key Skills

Education

Any Graduate