Description

You will design, implement, and manage scalable Kafka-based data pipelines and distributed systems.

Responsibilities

• Design and implement fault-tolerant Kafka data pipelines and microservices across DEV, QA, UAT, and PROD environments.
• Administer distributed Kafka clusters and troubleshoot messaging or performance issues within established timelines.
• Provide technical direction, guidance, and code reviews to other engineers on the project.
• Collaborate with business and IT teams to translate requirements into technical solutions using Agile methodology.
• Apply an automate-first approach to subsystem implementation and debugging.

Required Skills

• 8+ years of experience in distributed systems and data engineering.
• Deep expertise in Confluent Kafka, including producers, consumers, topics, partitions, brokers, and replication.
• Proficiency in Java or Scala; Python knowledge is a plus.
• Experience designing scalable architectures considering throughput, fault tolerance, and latency.
• Hands-on experience with microservices and data serialization formats like JSON, Avro, and Protobuf.
• Proven ability to manage data schema evolution.
• Experience deploying and managing Kafka on cloud platforms including AWS, Azure, or GCP.
• Strong troubleshooting skills for monitoring cluster health and identifying bottlenecks.
• Any Graduate degree.

Preferred Skills

• Knowledge of Kafka Streams API for real-time data processing.

Key Skills

Education

Any Graduate