Description

You will design and build scalable data pipelines on AWS for batch and near-real-time processing.

This role is hybrid.

Responsibilities

• Build ETL/ELT workflows using AWS Glue, orchestrated with Apache Airflow and/or AWS Step Functions.
• Develop data transformations using Python and PySpark, implementing complex SQL logic for analytics.
• Model data for analytics using dimensional modeling within Amazon Redshift.
• Implement AWS security and access controls using IAM to support reliable data operations.
• Apply DevSecOps practices and Infrastructure as Code (Terraform) for pipeline deployment.

Required Skills

• 7+ years of data engineering experience on AWS.
• Core AWS services: Glue, S3, Redshift, Athena, Lambda.
• Proficiency in Python and PySpark for data processing.
• Strong SQL skills with experience in dimensional modeling and data warehousing.
• Experience with orchestration tools like Apache Airflow or AWS Step Functions.
• Understanding of IAM security fundamentals.
• Familiarity with CI/CD and DevOps practices.

Preferred Skills

• AWS Certified Data Engineer or Solutions Architect certification.
• Experience with streaming technologies like Apache Kafka or Amazon Kinesis.
• Exposure to Databricks or Snowflake.

Key Skills

Education

Any Graduate