Description

You will build and deploy data pipelines within the AWS ecosystem.

Responsibilities

• Build and maintain data pipelines using PySpark and AWS Glue.
• Manage pipeline deployments using CloudFormation templates and Lambda instances.
• Configure messaging services including SQS and SNS for pipeline scheduling.
• Implement AWS security and networking protocols via IAM.
• Own the end-to-end deployment process from initial build to production.

Required Skills

• 5+ years of professional experience in data engineering.
• Strong proficiency in Python.
• Hands-on experience with Spark and PySpark.
• Expertise with AWS Glue and AWS Lambda.
• Experience with CloudFormation templates for infrastructure deployment.
• Deep understanding of AWS IAM, S3, SQS, and SNS.
• Proven ability to manage AWS security and networking requirements.
• Solid grasp of AWS cloud systems and the full deployment lifecycle.

Key Skills

Education

Any Graduate