Description

You will manage and optimize Kafka infrastructure and real-time data pipelines.

Responsibilities

• Configure Kafka topics, brokers, and ZooKeeper instances.
• Build real-time data ingestion pipelines using Kafka connectors for CRUD operations.
• Deploy Kafka nodes as PODs or Docker containers on AKS or OpenShift Kubernetes platforms.
• Implement error handling, retry logic, and message replay mechanisms.
• Monitor Kafka performance and set up dashboards using Grafana and Prometheus.

Required Skills

• 8+ years of experience as a Kafka Administrator.
• Hands-on experience with Python for scripting.
• Expertise in Confluent Kafka and Confluent Schema Registry.
• Experience streaming on-premises data for cloud analytics.
• Proven ability to optimize Kafka deployments for throughput, latency, durability, and availability.
• Experience managing Kafka on Kubernetes (AKS/OpenShift).
• Proficiency with Prometheus and Grafana for monitoring.

Preferred Skills

• Experience setting up and managing schedulers for data workflows.

Key Skills

Education

Any Graduate