Description

You will build and maintain large-scale data pipelines using PySpark and ETL processes, ensuring efficient data processing and transformation logic.

This role is on-site.

Responsibilities

• Develop and optimize ETL workflows for high-throughput data processing.
• Implement complex data transformation logic using PySpark.
• Maintain existing data pipelines and troubleshoot processing issues.
• Ensure data integrity and performance across large datasets.
• Collaborate with the team in the NY office.

Required Skills

• 10+ years of experience in data engineering or related roles.
• Expert-level proficiency in PySpark for large-scale data processing.
• Proven track record of building and managing ETL pipelines.
• Bachelor's degree or equivalent graduate-level education.
• Ability to work onsite in the NY office 1-2 days per week.

Key Skills

Education

ANY GRADUATE