Description

You will design and develop streaming pipelines and data platforms for high-volume batch and real-time processing.

This role is on-site.

Responsibilities

• Design and implement real-time streaming pipelines using Kafka for high-throughput, fault-tolerant systems.
• Develop producers and consumers for event-driven architectures and distributed systems.
• Debug and fix post-production issues to ensure system reliability.
• Lead technical design and architecture activities while spending 70% of your time in coding and 30% in team mentoring and coordination.

Required Skills

• 5+ years of extensive experience in designing and developing distributed systems, streaming pipelines, and data platforms.
• Strong expertise in Java, Kafka, Spark, and Hadoop.
• Proficiency with Scala, Cassandra, Azure, PySpark, SQL, and NoSQL databases.
• Experience working in Agile environments and with cloud platforms.

Preferred Skills

• Experience with Hive, YARN, HDFS, Linux, CI-CD with Jenkins and Github, and observability tools like Dynatrace.
• Knowledge of orchestration tools like Airflow, NiFi, and Griffin, plus data engineering tools like Pandas and Pola.rs.
• Functional experience in Retail Banking, capital markets, and securities processing.

Key Skills

Education

Any Graduate