Description

Architect and implement cloud-native data pipelines on AWS, owning data quality, monitoring, and schema design.

This role is on-site.

Responsibilities

• Architect and implement cloud-native data pipelines on AWS using serverless compute layers.
• Design data lake storage strategies and lead schema design for Athena and Postgres tables, including DDL, partitioning, and indexing.
• Build and debug Airflow production DAGs, integrating real-time data streams via Kinesis and SQS.
• Collaborate with business analysts to translate reporting requirements into schema and pipeline design.
• Evaluate and integrate AI/ML services and maintain architecture runbooks.

Required Skills

• 8+ years of experience in data engineering.
• Expert proficiency in Python, including boto3 and async processing.
• Deep knowledge of AWS services: S3, Lambda, SQS, Kinesis, Step Functions, EventBridge, DynamoDB, Glue, Athena, and Redshift.
• Hands-on experience with Airflow production DAG development and debugging.
• Strong SQL skills for complex queries, window functions, and performance tuning.
• Experience with IAM roles, policies, cross-account access, and permissions boundaries.
• Bachelor's degree in a relevant field.

Key Skills

Education

Bachelor's degree