Description

Build and support enterprise data pipelines within a Hadoop environment to deliver data assets for finance and risk transformation.

This role is on-site.

Responsibilities

• Build and maintain data pipelines using an in-house framework to extract and transform Hadoop data.
• Aggregate data for downstream consumption across various organizational units.
• Perform performance tuning and optimization within Spark processes.
• Support the delivery of high-visibility data assets specifically for financial transformation.

Required Skills

• 4+ years of experience in data engineering or big data development.
• Hands-on experience with Python and PySpark scripting.
• Proficiency with Spark SQL and Hadoop environments.
• Strong background in ETL processes and data aggregation.
• Experience working within the finance sector.
• Proven ability to perform performance tuning and optimization in Spark.
• Bachelor's degree or equivalent graduate qualification.

Preferred Skills

• Experience with Java for working on application components.
• Knowledge of Ab Initio for migrating workloads to Spark.

Key Skills

Education

Any Graduate