Description

Lead data engineering initiatives on AWS, focusing on ELT architecture and automated orchestration.

This role is on-site.

Responsibilities

• Orchestrate data workflows using Amazon MWAA to manage ingestion, transformation, and integrity tasks.
• Transition ETL processes to an ELT approach, implementing metadata-driven integration for rapid system onboarding.
• Build scalable data pipelines using pySpark, Python, SQL, and Informatica within a Data Lakehouse architecture.
• Implement CI/CD pipelines via GitHub Actions and manage Infrastructure as Code using Terraform.
• Deploy event-driven architectures with AWS EventBridge and secure environments using S3, Redshift, and Snowflake RBAC.

Required Skills

• 8+ years of professional experience in data engineering or related fields.
• Expertise in Amazon Managed Workflows for Apache Airflow (MWAA).
• Proficiency in pySpark, Python shell, and SQL for pipeline development.
• Hands-on experience with Informatica and ELT migration strategies.
• Strong command of Infrastructure as Code (Terraform) and CI/CD (GitHub Actions).
• Deep understanding of AWS services: S3, Redshift, EventBridge, and CloudWatch.
• Experience integrating Snowflake for enhanced RBAC and observability.

Key Skills

Education

Any Graduate