Description

Design and build scalable data pipelines and architectures within a Big Data Hadoop environment. You will own the release pipeline, enforce engineering standards, and provide technical leadership.

Responsibilities

• Develop secure, scalable data pipelines using Spark on Hadoop or object storage.
• Architect repeatable data flow schemes in Hadoop to eliminate manual steps.
• Automate data ingestion, movement, and access workflows.
• Debug production incidents, perform root cause analysis, and implement mitigation plans.
• Collaborate with business stakeholders and engineering teams to build scalable products.

Required Skills

• 10+ years of experience in Data Warehouse projects.
• Expertise in implementing end-to-end DW projects in Hadoop environments.
• Proficiency in building data pipelines using Spark with Python.
• Strong SQL knowledge and experience with Oracle and Hadoop databases.
• Experience working within Agile teams.
• Proven ability to automate data flow processes in Big Data environments.
• Strong analytical skills for debugging and troubleshooting production issues.
• Bachelor's degree or equivalent graduate qualification.

Preferred Skills

• Experience with real-time data flow systems using NiFi and Kafka.

Key Skills

Education

Any Gradute