Description

You will build and optimize large-scale data pipelines on AWS.

This role is remote.

Responsibilities

• Build, maintain, and optimize data pipelines using Spark.
• Migrate existing CMS data platforms into AWS environments.
• Develop libraries to standardize data processing workflows.
• Implement automation for applicable data processes.
• Collaborate with business analysts and architects to design technical architectures.

Required Skills

• 5+ years of experience in a data engineering role.
• Proficiency in Python and SQL.
• Strong experience building data pipelines with Spark.
• Hands-on experience with AWS EMR and Glue.
• Experience with relational, NoSQL, and cloud object stores.
• Legal right to work in the USA.

Preferred Skills

• Experience with Terraform or other infrastructure as code solutions.
• Production experience with ACID compliant formats such as Hudi, Iceberg, or Delta Lake.

Key Skills

Education

Any Graduate