Description
You will manage and optimize Kafka infrastructure and data streaming pipelines.
Responsibilities
- Provision Kafka topics, brokers, and ZooKeeper instances.
- Configure real-time data ingestion from source to sink using Kafka connectors for CRUD operations.
- Manage streaming on-premises data for cloud analytics using Confluent Kafka.
- Implement error handling and message retry/replay logic using error topics and queues.
- Deploy Kafka nodes as PODs or Docker containers on AKS and OpenShift managed Kubernetes platforms.
Required Skills
- 5-8 years of experience as a Kafka Administrator.
- Proficiency in Python for scripting and automation.
- Strong Shell Scripting skills.
- Experience setting up Confluent Schema Registry.
- Ability to optimize Kafka deployments for throughput, latency, durability, and availability.
- Hands-on experience with Docker.
- Experience with Kubernetes platforms including AKS and OpenShift.
- Knowledge of Prometheus and Grafana for monitoring dashboard setup.
- Any Graduate degree.