Description

You will design and manage Kafka infrastructure to support data pipelines across the organization.

Responsibilities

• Consult with engineering teams to integrate Kafka into their existing data pipelines.
• Architect, build, and support new and existing Kafka clusters using Infrastructure as Code.
• Partner with Splunk teams to route traffic through Kafka using open-source agents and collectors deployed via Chef.
• Remediate Kafka health issues and automate operational processes where possible.
• Manage CI/CD pipelines, create monitoring dashboards, and perform PoCs on new components to enhance Kafka offerings.

Required Skills

• 5+ years of experience with Kafka architecture and management.
• Hands-on experience with Chef for configuration management and deployment.
• Experience routing traffic and integrating with Splunk.
• Proficiency in managing CI/CD pipelines.
• Experience using Infrastructure as Code (IaC) to build and support clusters.
• Ability to develop monitoring dashboards and alerting systems.
• Experience conducting Proof of Concepts (PoC) for new technical components.

Key Skills

Education

Any Graduate