Description

You will build and manage cloud-native data solutions within an AWS-based Data Mesh architecture.

This role is on-site.

Responsibilities

• Design and implement AWS cloud-native solutions for Ingestion as a Service modules.
• Build Spark-based frameworks to support scalable and reusable data ingestion pipelines.
• Develop, build, and test data engineering pipelines using Python and PySpark.
• Operate and manage CI/CD pipelines using Jenkins and CloudFormation.
• Apply functional programming to develop modular microservices and assist with conceptual data modeling.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency with Python and PySpark.
• Hands-on experience with Apache Spark.
• Experience with AWS services including Glue, Lambda, Step Functions, DynamoDB, RDS (Postgres), and Lake Formation.
• Knowledge of CI/CD pipelines, Jenkins, and CloudFormation.
• Understanding of data management concepts: Master Data Management, Critical Data Elements, Granularity, and Normalization.
• Ability to work with Event Triggers and AWS cloud-native architectures.

Preferred Skills

• Any Graduate degree.

Key Skills

Education

Any Graduate