Description

You will design, implement, and maintain highly available Kafka clusters in production. You own the full lifecycle of the platform, from provisioning and security to monitoring and incident resolution.

Responsibilities

• Design and maintain Kafka clusters, performing upgrades and patching with minimal downtime.
• Administer security protocols, access controls, ACLs, and Schema Registry.
• Monitor cluster health and performance using Prometheus and Grafana, troubleshooting broker outages and replication lag.
• Develop automation scripts and Infrastructure as Code (Ansible, Terraform) to streamline provisioning and configuration.
• Manage Kafka Connect, Streams, and Mirror Maker for data pipelines, ensuring platform reliability on Linux systems.

Required Skills

• 5+ years of Kafka administration experience, ideally with Confluent Kafka.
• Deep knowledge of Kafka ecosystem components: ZooKeeper, KRaft, Schema Registry, MirrorMaker.
• Proficiency in Linux (RHEL, Ubuntu) and scripting with Python or Bash.
• Experience with Infrastructure as Code tools like Ansible and Terraform.
• Hands-on knowledge of monitoring tools, specifically Prometheus and Grafana.
• Experience deploying Kafka clusters using OpenShift, Kubernetes, Helm charts, and Docker images.
• Understanding of Kafka resiliency, DC/DR setups, and observability.

Preferred Skills

• Experience with data ingestion and streaming pipeline setup for platforms like Solr.
• Ability to build custom Kafka tooling and dashboards for producers and consumers.

Key Skills

Education

Bachelor's degree