Description

You will build and manage data pipelines across batch, streaming, and API ingestion patterns.

Responsibilities

• Design and implement data pipelines using AWS Glue and Airflow for batch and streaming workloads.
• Develop streaming data pipelines utilizing AWS MSK, AWS SQS, and Amazon Kinesis.
• Build API ingestion pipelines using AWS API Gateway.
• Collaborate with DevOps to deploy data engineering code through CI/CD pipelines like CodePipeline and CodeDeploy.
• Work with Solution Architects to evolve metadata-driven ingestion frameworks using AWS native services.

Required Skills

• 8-10 years of professional experience in data engineering roles.
• Extensive experience with AWS Glue (Spark) and Python scripting.
• Hands-on expertise with Airflow for workflow orchestration.
• Proficiency in streaming technologies including AWS MSK, AWS SQS, and Amazon Kinesis.
• Experience implementing API ingestion patterns via AWS API Gateway.
• Knowledge of CI/CD tools including AWS CodePipeline, CodeDeploy, and enterprise Git.
• Any Graduate degree.

Preferred Skills

• AWS Certified Data Analytics – Specialty certification.

Key Skills

Education

Any Graduate