Description

You will design, build, and maintain scalable data pipelines within an AWS ecosystem.

Responsibilities

• Develop and maintain data pipelines using Python and PySpark.
• Design and implement efficient SQL queries for data extraction and manipulation.
• Own architectural decisions and strategic vision for data workflows.
• Implement and manage AWS cloud services including Athena, S3, Lambda, and EventBridge.
• Ensure data quality and integrity through rigorous testing and validation.
• Collaborate cross-functionally to translate data requirements into technical solutions.

Required Skills

• 8+ years of experience in data engineering roles.
• Proficiency in Python and PySpark.
• Strong SQL skills for data transformation and analysis.
• Hands-on experience with AWS Glue and AWS Glue Data Catalog.
• Experience managing data in Aurora Postgres and Redshift.
• Working knowledge of Airflow or MWAA DAGs for orchestration.
• Experience with ECS, Docker, or distributed computing concepts like EMR.
• Familiarity with Informatica or similar ETL tools for data integration.
• Understanding of Lakehouse concepts and database consumption patterns.

Key Skills

Education

Any Gradute