Description

Build and manage end-to-end data pipelines within the AWS ecosystem.

Responsibilities

• Design and implement ETL/ELT processes for data ingestion, storage, integration, and processing.
• Build data pipelines to ingest, transform, and deliver data using S3, Glue, RDS, Redshift, and Athena.
• Manage data modeling, CDC, data governance, and security using IAM and security credentials.
• Deploy infrastructure using CloudFormation templates and maintain pipelines via CI/CD and Jenkins.
• Monitor system health and performance using CloudWatch.

Required Skills

• 8+ years of experience in data engineering roles.
• Proficiency in SQL, Python, and PySpark, including complex scripting and performance tuning.
• Hands-on experience with AWS Glue, S3, Redshift, RDS, Athena, and Lambda.
• Experience with Lake Formation, CloudFormation, Jenkins, and CI/CD.
• Working knowledge of IAM and data security protocols.
• Familiarity with Jira and Confluence for task management and documentation.
• AWS Certified Solutions Architect Associate or Developer Associate.

Preferred Skills

• Experience with streaming ingestion using Kafka, Kinesis, or Spark Streaming.
• Proficiency in Unix Shell Scripting.

Key Skills

Education

ANY GRADUATE