Description

You will design, develop, and maintain ETL pipelines and manage data solutions.

This role is hybrid.

Responsibilities

• Design and develop ETL pipelines using PySpark, Python, and SQL.
• Implement and manage data solutions leveraging AWS services (S3, Glue, EMR, Redshift, Lambda).
• Optimize data workflows for performance, scalability, and cost efficiency.
• Monitor, troubleshoot, and improve existing data pipelines and infrastructure.
• Collaborate with data scientists and analysts to deliver reliable datasets.

Required Skills

• 10+ years of experience as a Data Engineer.
• Strong proficiency in Python and PySpark for data processing.
• Advanced knowledge of SQL for querying and data modeling.
• Hands-on experience with AWS services (S3, Glue, EMR, Redshift, Lambda).
• Experience with workflow orchestration tools (Airflow, Step Functions).
• Proficiency with CI/CD pipelines (GitHub Actions, CodePipeline, Jenkins).
• Proficiency with Docker and Kubernetes.
• Solid understanding of distributed computing and data lake/warehouse concepts.

Preferred Skills

• Bachelor's or Master's degrees in Computer Science or related field.

Key Skills

Education

Bachelor's or Master's degrees