Description

Design and build scalable data pipelines and architectures within a Big Data Hadoop environment, owning the release pipeline and enforcing engineering standards.

This role is on-site.

Responsibilities

• Develop secure, scalable data pipelines using Spark with Python on Hadoop or object storage.
• Architect repeatable data flow schemes in Hadoop to eliminate manual steps and automate ingestion.
• Debug production incidents, perform root cause analysis, and implement mitigation plans.
• Collaborate with business stakeholders and engineering teams to build scalable products.

Required Skills

• 10+ years of experience in Data Warehouse projects.
• Expertise in implementing end-to-end DW projects in Hadoop environments.
• Proficiency in building data pipelines using Spark with Python.
• Strong SQL knowledge and experience with Oracle and Hadoop databases.
• Experience working within Agile teams.
• Proven ability to automate data flow processes in Big Data environments.
• Bachelor's degree or equivalent graduate qualification.

Preferred Skills

• Experience with real-time data flow systems using NiFi and Kafka.

Key Skills

Education

Any Gradute