Description

You will design, implement, and expand data pipelines to process complex datasets from various banks and processors.

Responsibilities

• Build and maintain ETL pipelines to ingest vast amounts of disorganized Big Data into our systems.
• Lead and mentor the ETL team to ensure project goals and deadlines are met.
• Investigate pipeline issues, notify end-users, and propose technical solutions.
• Develop data collection systems to optimize statistical efficiency and data quality.
• Analyze complex data sets to identify trends, patterns, and process improvement opportunities.

Required Skills

• Proficiency in at least one high-level programming language: Python, Java, JavaScript, or C++.
• Experience working with complex, unorganized Big Data sets.
• Familiarity with Kafka on-premises architecture and ELK.
• Understanding of cross-cluster replication, index lifecycle management, and hot-warm architectures.
• Ability to design ETL pipelines based on data pattern analysis.
• Bachelor’s degree in Computer Science, Electrical Engineering, or Information Technology.
• 1+ years of relevant experience.

Preferred Skills

• Experience with Hadoop, Spark, or other Big Data tools.
• Experience with HPCC and ECL.

Key Skills

Education

ANY GRADUATE