Description

Manage and optimize large-scale data pipelines and distributed processing systems within a high-availability banking environment. You will maintain Tier-1 data-driven applications ensuring 24/7 uptime and strict SLA adherence.

Responsibilities

• Build and manage data pipelines to aggregate and transform raw data from diverse sources.
• Optimize data delivery and redesign systems to improve performance and transformation workflows.
• Maintain processed data in parallel on distributed Hadoop storage using MapReduce.
• Identify technical bottlenecks and communicate effective solutions to the engineering team.

Required Skills

• 12+ years of experience maintaining applications using Java, J2EE, SDLC, and WebSphere.
• 6+ years of recent experience working for a major US bank or brokerage house.
• 6+ years of experience with Cassandra, Hadoop, MongoDB, Apache Spark, HDFS, YARN, MapReduce, Pig, Hive, Flume, Sqoop, and ZooKeeper.
• 6+ years of experience developing on RDBMS such as Microsoft SQL Server or PostgreSQL.
• 5+ years of programming experience in Scala, Java, Python, or Go.
• 5+ years of development experience on Hadoop/Spark.
• Proven experience handling large datasets, regularly transforming and querying sets exceeding 20 million records.
• Strong understanding of SOA design principles and database maintenance.

Preferred Skills

• Exposure to data hygiene routines and models.

Key Skills

Education

Bachelor's Degree