Description

You will build and manage complex data pipelines and migration processes using Python and AWS.

Responsibilities

• Develop and maintain data pipelines using Python and PySpark.
• Execute data migrations and manage large-scale data structures.
• Orchestrate workflows using Apache Airflow.
• Implement infrastructure as code using Terraform.
• Utilize AWS services including Glue and Lambda to support data engineering workflows.

Required Skills

• 7+ years of Python programming experience, specifically with complex data structures.
• 5+ years of experience with Python libraries including Pandas, PySpark, and Redis.
• 5+ years of experience using Apache Airflow for orchestration.
• Strong SQL programming skills, including advanced aggregate functions like SUM and AVG.
• Hands-on experience with AWS Glue, Lambda, and AWS ecosystem.
• Proficiency with Terraform for infrastructure management.
• Experience with Statistical Analysis Systems (SAS).
• Bachelor's degree in Computer Science.

Key Skills

Education

Bachelor's degree in Computer Science