Description

You will own the data platform features, pipeline optimization, and AWS migration efforts.

Responsibilities

• Build, maintain, and optimize data pipelines using Spark.
• Migrate CMS data platforms into AWS environments.
• Collaborate with business analysts and solutions architects to develop technical architectures for enterprise projects.
• Develop libraries to standardize data processing across the team.
• Implement automation for applicable data processes.

Required Skills

• 5+ years of experience in a data engineering position.
• Proficiency in Python and SQL.
• Strong experience building data pipelines with Spark.
• Experience building data processing infrastructure in AWS, including EMR or Glue.
• Experience with relational datastores, NoSQL datastores, and cloud object stores.
• Legal right to work in the USA.

Preferred Skills

• Experience with infrastructure as code using Terraform.
• Production experience with ACID compliant formats such as Hudi, Iceberg, or Delta Lake.
• Familiarity with data observability solutions or data governance frameworks.

Key Skills

Education

Any Graduate