Description

Build and maintain scalable data ingestion pipelines, data lakes, and ETL/ELT processes on AWS.

This role is hybrid.

Responsibilities

• Design and implement high-velocity streaming solutions using Amazon Kinesis and AWS Managed Kafka.
• Build automated data pipelines to ingest data from relational databases, file systems, and REST APIs into AWS data lakes and warehouses.
• Implement ETL/ELT processes and end-to-end data solutions including ingest, storage, integration, and processing.
• Develop and maintain CI/CD pipelines for the EDP Platform using CloudFormation and Jenkins.
• Administer DevOps tools, manage source code repositories, and ensure security compliance through RBAC strategies.

Required Skills

• 8+ years of experience as a Data Engineer on the AWS stack.
• Proficiency in Python, Shell scripting, and SQL, including stored procedures.
• Deep knowledge of AWS services: S3, Athena, Glue, EMR/Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, Kinesis, EventBridge, and IAM.
• Experience with Infrastructure as Code using CloudFormation.
• Strong background in DevOps practices, including Jenkins, CloudBees, and pipeline-as-code (Shared Libraries).
• Experience with data migration tools such as AWS DMS and DataSync.
• Bachelor's degree in Computer Science, Software Engineering, MIS, or equivalent.
• AWS Solutions Architect or AWS Developer Certification required.

Preferred Skills

• AWS Managed Airflow experience.
• Experience with containers (Docker) and modern source control systems like Bitbucket.

Key Skills

Education

Bachelor's degree