Description

Design, implement, and expand data pipelines to process complex datasets from banks and processors.

This role is on-site.

Responsibilities

• Build and maintain ETL pipelines to ingest large volumes of unstructured data.
• Lead and mentor the ETL team to meet project goals and deadlines.
• Investigate pipeline issues, notify end-users, and propose technical solutions.
• Develop data collection systems to optimize statistical efficiency and data quality.
• Analyze complex datasets to identify trends and process improvement opportunities.

Required Skills

• Proficiency in at least one high-level language: Python, Java, JavaScript, or C++.
• Experience working with complex, unorganized Big Data sets.
• Familiarity with Kafka on-premises architecture and ELK.
• Understanding of cross-cluster replication, index lifecycle management, and hot-warm architectures.
• Ability to design ETL pipelines based on data pattern analysis.
• Bachelor’s degree in Computer Science, Electrical Engineering, or Information Technology.
• 1+ years of relevant experience.

Preferred Skills

• Experience with Hadoop, Spark, or other Big Data tools.
• Experience with HPCC and ECL.

Key Skills

Education

ANY GRADUATE