Description

You will build, maintain, and optimize data pipelines to support a large-scale data platform in Columbus, OH.

Responsibilities

• Build and optimize data pipelines using Spark to consolidate data assets.
• Develop libraries to standardize data processing across the organization.
• Migrate CMS data platforms into AWS environments and implement automation for applicable data processes.
• Partner with business analysts and architects to design technical architectures for enterprise projects.
• Advise and coach other data professionals on engineering standards and practices.

Required Skills

• 5+ years of experience in a data engineering role.
• Strong proficiency in Python and SQL.
• Strong experience building data pipelines with Spark.
• Experience building data processing infrastructure in AWS, including tools like EMR or Glue.
• Experience with relational datastores, NoSQL datastores, and cloud object stores.
• Bachelor's degree in Computer Science, Programming, or a similar field.
• Strong analytical and problem-solving skills.
• Strong verbal and written communication skills.

Preferred Skills

• Experience with infrastructure as code, specifically Terraform.
• Production experience with ACID compliant formats such as Hudi, Iceberg, or Delta Lake.
• Familiarity with data observability solutions and data governance frameworks.

Key Skills

Education

Bachelor's degree in Computer Science