Description

Key Skills: Java, Kafka, Spark, Hadoop, Scala, Cassandra, Azure, PySpark, SQL, NoSQL

Good to Have Skills: Extensive experience with Hive, YARN, HDFS, Linux, CI-CD with Jenkins and Github, Observability tools like Dynatrace, Databricks, Azure Data Factory, ADLS, Kubernetes, Docker, ETL tools like Pandas and Pola.rs, Ibis, MongoDB, orchestration tools like NiFi, Griffin, Hamilton, Airflow, and functional experience in Retail Banking, capital markets, and securities processing.

Roles & Responsibilities:

• Design and develop streaming pipelines and data platforms to handle high volume data processing for both batch and real-time systems.
• Design and implement real-time streaming pipelines using Kafka for high-throughput, fault-tolerant systems.
• Develop producers and consumers for event-driven architectures and distributed systems.
• Lead technical design and architecture activities while spending 70% time in coding and 30% in team mentoring and coordination.
• Debug and fix post-production issues for developed code and ensure system reliability.
• Work with cloud platforms and orchestration tools to build scalable data processing solutions.
• Collaborate in Agile environments and maintain excellent written and verbal communication with stakeholders.

Experience Required: Extensive experience in designing and developing distributed systems, streaming pipelines, and data platforms. Strong expertise required in debugging and fixing post-production issues. Experience working in Agile environments and with cloud platforms preferred

Key Skills

Education

Any Graduate