Description

Design, implement, and maintain scalable Kafka-based data streaming pipelines.

This role is on-site.

Responsibilities

• Design and build scalable Kafka data pipelines focused on throughput, fault tolerance, and latency.
• Implement and manage streaming pipelines using Confluent Kafka architecture.
• Troubleshoot cluster health, resolving broker failures, consumer lag, and message delivery issues.
• Diagnose performance bottlenecks related to network congestion, disk I/O, or throughput limits.
• Manage data schema evolution and serialization formats while ensuring security mechanisms.

Required Skills

• 5+ years of experience in Kafka development and messaging systems.
• Deep understanding of Confluent Kafka architecture (brokers, topics, partitions, consumer groups).
• Proficiency in Java or Scala; familiarity with Python.
• Strong experience with Kafka Connect and KSQL.
• Hands-on experience with cloud platforms: AWS, Azure, or GCP.
• Knowledge of data serialization formats: JSON, Avro, Protobuf.
• Experience with Kafka security: authentication, authorization, and encryption.
• Solid understanding of distributed systems concepts (leader election, fault tolerance).

Preferred Skills

• Knowledge of Kafka Streams API for real-time data processing.

Key Skills

Education

Any Gradute