Description

You will architect, build, and maintain event-driven data pipelines and scalable data lake structures using AWS services.

This role is on-site.

Responsibilities

• Architect and maintain event-driven pipelines using AWS Kinesis, MSK/Kafka, Lambda, Step Functions, SQS/SNS, and Glue/EMR.
• Develop ETL/ELT workflows using Python and PySpark, optimizing for performance, scalability, and cost efficiency.
• Design and manage data lake and warehouse structures using S3, Glue Catalog, Athena, and Redshift.
• Implement streaming solutions with checkpointing, stateful transformations, idempotency, and schema evolution.
• Establish data quality standards, observability, monitoring, and security best practices including IAM and KMS.

Required Skills

• 15+ years of experience in data engineering or data architecture.
• Deep expertise in AWS services: Kinesis, MSK/Kafka, Lambda, Step Functions, SQS/SNS, Glue, EMR, S3, Athena, Redshift.
• Strong proficiency in Python and PySpark for ETL/ELT development.
• Experience with data lake architecture, partitioning strategies, and data modeling.
• Knowledge of CI/CD pipelines and automated deployment frameworks.
• Familiarity with monitoring and alerting tools such as CloudWatch and Datadog.
• Understanding of data security, governance, and IAM policies.
• Bachelor's degree in Computer Science or related field.
• Ability to lead design reviews and mentor junior engineers.

Preferred Skills

• Experience with Java in data engineering contexts.
• Proven track record of collaborating with data scientists and business teams.

Key Skills

Education

Bachelor's degree