Description

You will build and maintain large-scale data processing pipelines using distributed systems. You own the development, testing, and deployment of batch and real-time data solutions.

This role is on-site.

Responsibilities

• Develop and optimize data pipelines using Apache Spark Core, SQL, and Streaming.
• Implement real-time data ingestion and processing using Kafka.
• Manage Hadoop ecosystem components including HDFS and Hive.
• Write scalable data processing code in Python, Scala, or Java.
• Maintain strong SQL proficiency and Linux/Unix command-line operations.

Required Skills

• 5+ years of experience with Apache Spark (Core, SQL, Streaming).
• Hands-on experience with Kafka and real-time data processing.
• Proficiency in Hadoop, HDFS, and Hive.
• Strong programming skills in Python, Scala, or Java.
• Advanced SQL skills and Linux/Unix environment familiarity.
• Bachelor's degree in Computer Science or related field.

Preferred Skills

• Experience with Fraud Detection, AML, or Transaction Monitoring.
• Knowledge of Banking and regulatory domain requirements.
• Familiarity with AWS or Azure cloud platforms and NoSQL databases (HBase, Cassandra, MongoDB).

Key Skills

Education

Bachelor's degree