Description

Lead data engineering efforts and architect scalable data pipelines within an AWS environment.

Responsibilities

• Design and deploy AWS services using CloudFormation and CDK stacks in Python.
• Build and maintain real-time data integration using Kinesis data streams and Firehose delivery streams.
• Develop and integrate REST APIs using Python frameworks like Flask or Django.
• Manage data lake architectures utilizing AWS Glue, Athena, and Lake Formation.
• Debug complex code and architect solutions that account for edge case scenarios in micro-services.

Required Skills

• 12+ years of overall Data Engineering experience, including 3-4 years in a lead capacity.
• 5+ years of Python application development experience.
• 2+ years of hands-on experience with AWS services including Lambda, EventBridge, CloudWatch, S3, DynamoDB, Kinesis, CloudFormation, and Systems Manager.
• 2+ years of experience deploying services via AWS CDK using Python.
• 2+ years of experience with REST API integration using Flask, Django, or similar frameworks.
• 2+ years of hands-on Data Lake experience using AWS Lake Formation, Glue, and Athena.
• 2+ years of experience with Kinesis data streams and Firehose delivery streams.
• 3+ years of experience with DynamoDB and SQL.
• Practical experience working within micro-services architectures.

Preferred Skills

• Experience with AWS Lake Formation and Glue for data governance.
• Deep knowledge of event-driven architectures using EventBridge.

Key Skills

Education

Not specified