Description

You will build and maintain data pipelines on AWS, focusing on automation, quality, and observability.

Responsibilities

• Develop configuration-driven ingestion and transformation pipelines using AWS Glue, Lambda, and Redshift.
• Implement CI/CD pipelines for automated build, test, and deployment of data products.
• Establish automated test frameworks for schema validation, data quality, and performance testing.
• Set up monitoring and observability dashboards for product-level metrics using CloudWatch and custom metrics.
• Collaborate with data modelers and architects to ensure alignment with business requirements.

Required Skills

• 5+ years of experience in data engineering and pipeline development.
• Proficiency in Python, SQL, and shell scripting.
• Hands-on experience with AWS services: Glue, Redshift, S3, Lambda, and CloudWatch.
• Experience with CI/CD tools including Terraform, Jenkins, or AWS CodePipeline.
• Strong knowledge of ETL/ELT processes and frameworks.
• Bachelor’s degree in Computer Science, Data Engineering, or related field.

Key Skills

Education

Bachelor's degree