Description

Design and develop streaming pipelines and data platforms for high-volume batch and real-time processing.

This role is on-site.

Responsibilities

• Design and implement real-time streaming pipelines using Kafka for on-premises systems with high throughput and fault tolerance.
• Develop producers and consumers for distributed systems, ensuring optimal performance and reliability.
• Implement Kafka-based event-driven architectures to support complex data processing workflows.
• Debug and fix post-production issues, providing strong expertise in system stability.
• Lead technical initiatives including design, architecture, and coding, while mentoring the team.

Required Skills

• 5+ years of experience with Kafka and NoSQL databases.
• Hands-on experience with Cassandra, Spark, Scala, and Hadoop.
• Proficiency in MongoDB, ETL processes, and SQL.
• Strong understanding of distributed systems and system architecture concepts.
• Experience with Linux on-premises environments.
• Ability to work effectively in Agile environments with cross-functional teams.

Preferred Skills

• Strong expertise in Hadoop ecosystem components (Hive, YARN, HDFS).
• Experience with CI/CD tools (Jenkins, GitHub) and orchestration platforms (Airflow, NiFi).
• Functional experience in Retail Banking, capital markets, or securities processing.

Key Skills

Education

Any Graduate