Description

You will lead data engineering initiatives within the DAP Core Delivery Squad, supporting service transitions and migrating new lines of business to the AWS DAP Platform.

Responsibilities

• Lead technical initiatives to implement data engineering work within the delivery squad.
• Implement Apache Iceberg as the standard data lake storage format, managing partitioning and data organization across nodes.
• Collaborate with the Product Owner to transition DAP Core from a tier 3 to a tier 2 service.
• Review environments to scale down pre-production usage while improving code quality.
• Assist the Delivery Manager with technical documentation, project plans, requirements, and RAID logs.

Required Skills

• 5+ years of experience in data engineering.
• In-depth experience with AWS services: Amazon S3, Redshift, Glue, EMR, Lambda, Kinesis, Athena, and DynamoDB.
• Proficiency in designing and optimizing data warehousing solutions, schema design, and ETL/ELT processes.
• Hands-on experience with big data frameworks including Apache Spark, Hadoop, and Kafka.
• Strong SQL skills for Amazon Redshift and RDS, plus NoSQL experience with Amazon DynamoDB.
• Experience building scalable data pipelines using AWS Glue, AWS Data Pipeline, or serverless AWS Lambda solutions.
• Proficiency in Python scripting and experience with Java or Scala for data processing applications.
• Ability to integrate data from on-premises and cloud sources using AWS DMS or Glue.
• Knowledge of data security, encryption, access controls, and auditing within AWS.

Preferred Skills

• Understanding of cloud architecture principles and networking concepts to design scalable solutions.

Key Skills

Education

Any Graduate