Description

Design, deploy, and maintain scalable Kafka clusters while supporting the underlying Linux infrastructure for real-time data platforms.

This role is remote.

Responsibilities

• Design and maintain highly available Kafka clusters, including upgrades, patching, and security protocols.
• Monitor cluster health using Prometheus and Grafana, and resolve incidents like broker outages or replication lag.
• Develop Kafka topics, partitions, ACLs, and optimize throughput using Schema Registry and KRaft.
• Automate provisioning, configuration, and monitoring using Infrastructure as Code tools like Terraform and Ansible.
• Manage Kafka Connect, Kafka Streams, and Mirror Maker to support data pipelines and ensure platform reliability.

Required Skills

• 5+ years of Kafka administration experience, ideally with Confluent Kafka.
• Deep knowledge of Kafka ecosystem components: ZooKeeper, KRaft, Schema Registry, MirrorMaker.
• Proficiency in Linux (RHEL, Ubuntu), Python, and Bash scripting for automation.
• Hands-on experience with Ansible and Terraform for IaC and configuration management.
• Experience deploying Kafka on OpenShift/Kubernetes using Helm charts and Docker images.
• Strong understanding of Kafka resiliency, DC/DR setups, and observability.
• Ability to build custom tooling and dashboards for producers and consumers.

Preferred Skills

• Experience with data ingestion pipelines connecting Kafka to data sources like Solr.
• Background in BE/B Tech/MCA with strong communication skills.

Key Skills

Education

Any Graduate