Description

Build and maintain data pipelines and infrastructure automation within a Confluent Kafka ecosystem.

Responsibilities

• Develop Python scripts for data ingestion, transformation, and processing.
• Write and optimize Python code to ingest data from various sources into Kafka topics.
• Build Python-based Kafka consumers and producers to enable efficient data processing.
• Automate infrastructure provisioning using Python and Shell scripting.
• Manage Kafka topics, producers, consumers, and schemas.

Required Skills

• 5 to 7 years of experience in Python development.
• Proficiency in Shell scripting for automation.
• Deep understanding of Confluent Kafka architecture, including Brokers, Schema Registry, ZooKeeper, and Cluster linking.
• Experience developing Kafka consumers and producers.
• Knowledge of Kafka security mechanisms such as SASL, JAAS, Kerberos, and OAuth.
• Experience with Mirror-Making and Shared Clusters.
• Bachelor's Degree in any field.

Preferred Skills

• Experience monitoring and fine-tuning Kafka cluster configurations for performance and resource utilization.

Key Skills

Education

Any Graduate