Description

You will design, implement, and expand data pipelines by performing extraction, transformation, and loading activities.

Responsibilities

• Build ETL pipelines to ingest vast amounts of data from various banks and processors.
• Analyze complex, unorganized Big Data to identify patterns and clean datasets.
• Investigate pipeline issues, notify end-users, and propose technical solutions.
• Develop data collection systems to optimize statistical efficiency and data quality.
• Create reports to provide insights and document processes for future reference.

Required Skills

• Experience working with complex, large-scale datasets.
• Proficiency in at least one high-level programming language: Python, Java, JavaScript, or C++.
• Familiarity with Kafka on-premises architecture and ELK.
• Understanding of cross-cluster replication, index lifecycle management, and hot-warm architectures.
• Ability to interpret trends and patterns within complex data structures.
• Bachelor’s degree in Computer Science, Electrical Engineering, or Information Technology.

Preferred Skills

• Experience with Hadoop, Spark, or other Big Data tools.
• Experience with HPCC and ECL.

Key Skills

Education

ANY GRADUATE