Description

You will lead large-scale data processing initiatives, owning the development and optimization of Spark-based pipelines.

This role is on-site.

Responsibilities

• Develop and optimize Spark-based data processing pipelines for large datasets.
• Collaborate with data engineers and scientists to design robust data solutions.
• Write efficient, scalable code for distributed data processing.
• Monitor Spark applications, troubleshoot performance issues, and ensure data quality.
• Implement and enforce best practices in Spark development and ETL processes.

Required Skills

• 10+ years of experience in data engineering or related fields.
• Deep expertise with Apache Spark and distributed systems.
• Proficiency in Scala and Java for data processing.
• Strong experience with Hive and ETL processes.
• Knowledge of Hadoop ecosystem and data integration patterns.
• Proven ability in performance tuning and optimization.
• Bachelor's degree in Computer Science or related field.

Preferred Skills

• Experience in the Banking and Financial Services industry.
• Knowledge of capital management, liquidity management, and payments processes.

Key Skills

Education

Bachelor's degree