Description
You will manage and enhance Apache Kafka and Confluent Platform environments on AWS.
Responsibilities
- Manage and improve existing Apache Kafka and Confluent Platform implementations on AWS.
- Implement and maintain Kafka producers, consumers, Connectors, and Kafka Streams applications.
- Enforce governance for topic design, schema evolution, partitioning, and data retention.
- Monitor, troubleshoot, and tune Kafka clusters using Confluent Control Center, Prometheus, and Grafana.
- Automate Kafka infrastructure deployment and scaling using Kubernetes and Terraform.
Required Skills
- 8+ years in platform engineering with 5+ years of hands-on experience with Apache Kafka and Confluent Kafka.
- Proficiency in Kafka client development using Java or Python.
- Expertise with Confluent Platform, including Brokers, Schema Registry, Control Center, and ksqlDB.
- Experience deploying and managing Kafka on AWS via MSK or self-managed EC2 setups.
- Solid understanding of Kubernetes, specifically EKS, for microservices integration.
- Hands-on experience with Kafka Connect, Kafka Streams, and schema management.
- Infrastructure automation experience using Terraform and Helm.
- Experience with monitoring and alerting stacks such as Prometheus, Grafana, and ELK.
- Any Graduate degree.
Preferred Skills
- Experience in FinTech or other regulated industries.
- Knowledge of security best practices including TLS, SASL, OAuth, RBAC, and encryption at rest.
- Exposure to Apache Flink, Spark Streaming, or other stream processing engines.
- Experience establishing Kafka governance frameworks and multi-tenant topic strategies.