Description

You will manage and enhance the existing Apache Kafka and Confluent Platform deployed on AWS.

Responsibilities

• Review current Kafka implementations and recommend necessary improvements.
• Collaborate with engineering and product teams to integrate new use cases and define scalable streaming patterns.
• Implement and maintain Kafka producers/consumers, Connectors, and Kafka Streams applications.
• Enforce governance regarding topic design, schema evolution, partitioning, and data retention.
• Monitor, troubleshoot, and tune Kafka clusters using Confluent Control Center, Prometheus, and Grafana.
• Use Kubernetes and Terraform to automate Kafka infrastructure deployment and scaling, ensuring high availability and security.

Required Skills

• 10+ years of professional experience.
• Expertise with Apache Kafka and Confluent Kafka.
• Proficiency in Kafka client development using Python or Java.
• Experience deploying and managing Kafka on AWS, including MSK or self-managed EC2 setups.
• Solid understanding of Kubernetes, specifically EKS, for microservices integration.
• Hands-on experience with Kafka Connect, Kafka Streams, and schema management.
• Infrastructure automation experience using Terraform.
• Familiarity with monitoring stacks like Prometheus and Grafana.

Key Skills

Education

Bachelor's degree