Description

Build and maintain data warehousing and data lake applications within an Agile environment.

Responsibilities

• Develop PySpark data ingestion modules for cleansing, validation, curation, and aggregation.
• Build data processing workflows using AWS Glue, Step Functions, and AWS Lambda.
• Implement data pipelines using Python to ensure data integrity across the lake.
• Work within Agile methodologies to deliver incremental data engineering improvements.

Required Skills

• 5+ years of experience in data engineering roles.
• Proficiency in PySpark for data ingestion and processing.
• Hands-on experience with AWS Glue.
• Experience using AWS Lambda and Step Functions for orchestration.
• Strong Python programming skills for data processing tasks.
• Practical knowledge of Data Warehousing architectures.
• Experience managing Data Lake applications.
• Ability to work effectively in an Agile development lifecycle.

Preferred Skills

• Bachelor's degree in any field.

Key Skills

Education

Any Graduate