Description

You will lead data engineering initiatives within the DAP Core Delivery Squad to support the transition of services and the migration of new lines of business to the AWS DAP Platform.

Responsibilities

• Lead technical initiatives to implement data engineering work within the delivery squad.
• Collaborate with the Product Owner on the Planview initiative to transition DAP Core from a tier 3 to a tier 2 service.
• Implement Apache Iceberg as the standard data lake storage format, managing partitioning and data organization across nodes.
• Review environments to scale down pre-production usage while improving code quality.
• Assist the Delivery Manager with technical documentation, project plans, requirements, and RAID logs.

Required Skills

• 5+ years of experience in data engineering.
• In-depth experience with AWS services including Amazon S3, Redshift, Glue, EMR, Lambda, Kinesis, Athena, and DynamoDB.
• Proficiency in designing and optimizing data warehousing solutions, schema design, and ETL/ELT processes.
• Hands-on experience with big data frameworks including Apache Spark, Hadoop, and Kafka.
• Strong SQL skills for working with Amazon Redshift and RDS, and NoSQL experience with Amazon DynamoDB.
• Experience building scalable data pipelines using AWS Glue, AWS Data Pipeline, or serverless AWS Lambda solutions.
• Proficiency in Python scripting and experience with Java or Scala for data processing applications.
• Ability to integrate data from on-premises and cloud sources using AWS DMS or Glue.
• Knowledge of data security, encryption, access controls, and auditing within AWS.
• Experience using Amazon CloudWatch and CloudTrail for monitoring and performance optimization.

Preferred Skills

• Understanding of cloud architecture principles and networking concepts to design scalable solutions.

Key Skills

Education

Any Graduate