Description

You will build and support data pipelines for a growing application and its expanding use cases.

Responsibilities

• Analyze data structures and gather technical requirements to determine optimal data flow and destination.
• Design and build end-to-end data pipelines within existing infrastructure and frameworks.
• Perform QA work on all newly developed data pipelines to ensure data integrity.
• Integrate code into CI/CD pipelines and manage the pull request process.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency with Python.
• Hands-on experience with AWS services including Lambda, Glue, and S3.
• Experience using PySpark for data processing.
• Working knowledge of AI implementations.
• Ability to work within CI/CD workflows.
• Bachelor's degree or equivalent experience.

Key Skills

Education

Any Graduate