Description

You will develop and optimize ETL pipelines using PySpark and Databricks to support data migration projects. You will assist in migrating legacy on-premise applications from Informatica to cloud-based solutions. You will cleanse and transform unstructured and structured data from multiple vendors, including Bloomberg. You will implement data rules based on business requirements to create accurate and reliable data models. You will collaborate with team members in an Agile environment.

Responsibilities

• Develop and optimize ETL pipelines using PySpark and Databricks.
• Migrate legacy on-premise applications from Informatica to cloud-based solutions.
• Cleanse and transform data from multiple vendors, including Bloomberg.
• Implement business rules to create accurate data models.
• Collaborate in an Agile environment to accelerate development.

Required Skills

• Proficiency in PySpark with a focus on data processing and pipeline development.
• Experience with Databricks workflows and cloud-based data environments.
• Strong understanding of data cleansing, hierarchy, and applying business rules.
• Background in development, data modeling, or related fields.
• Ability to work effectively in a small, collaborative team.
• 5+ years of experience.

Key Skills

Education

Any Graduate