Description

Design, build, and maintain efficient data ingestion pipelines, data lakes, and warehousing solutions on AWS. You will implement high-velocity streaming, orchestrate workflows, and establish CI/CD pipelines for the enterprise data platform.

This role is hybrid.

Responsibilities

• Design and implement end-to-end data solutions including ingestion, storage, integration, processing, and access on AWS.
• Build automated data pipelines to ingest data from relational databases, file systems, and REST APIs into S3, RDS, Aurora, and Redshift.
• Implement high-velocity streaming solutions and orchestration using Amazon Kinesis, AWS Managed Airflow, and AWS Managed Kafka.
• Build and maintain CI/CD pipelines for the EDP Platform using CloudFormation and Jenkins, integrating SonarQube and security scans.
• Ensure performance and quality of high-scale data applications through unit testing, code reviews, and system design discussions.

Required Skills

• 8+ years of experience as a Data Engineer on the AWS stack.
• Deep expertise with AWS services: S3, Athena, Glue, Glue DataBrew, EMR/Spark, RDS, Redshift, DataSync, DMS, DynamoDB, Lambda, Step Functions, IAM, KMS, EventBridge, EC2, SQS, SNS, LakeFormation, CloudWatch, and CloudTrail.
• Strong programming experience with Python, Shell scripting, and SQL, including stored procedures.
• Proficiency in Infrastructure as Code using CloudFormation.
• Experience building CI/CD pipelines using Jenkins (Pipeline as Code, Shared Libraries), CloudBees, Maven, MS Build, and Gradle.
• Solid experience building solutions on AWS data lakes and data warehouses.
• Knowledge of implementing ETL/ELT processes and RBAC strategies using AWS IAM and Redshift RBAC.
• Experience with DevOps tools including Bitbucket, SonarQube, Artifactory, Docker, and Atlassian suite (Jira, Confluence).

Preferred Skills

• AWS Solutions Architect or AWS Developer Certification.
• Experience with AWS Managed Kafka.

Key Skills

Education

Bachelor's degree