Description

Build and automate data pipelines within the Confluent Kafka ecosystem using Python and Shell scripts.

This role is on-site.

Responsibilities

• Write and optimize Python code for data ingestion, transformation, and processing.
• Develop custom Kafka producers and consumers to facilitate efficient data movement.
• Automate infrastructure provisioning and management using Python and Shell scripts.
• Manage Kafka topics, schemas, and cluster configurations to optimize resource utilization.
• Implement access controls and authorization mechanisms including SASL, Kerberos, and OAuth.

Required Skills

• 5+ years of experience in Python development.
• Proficiency in writing Shell scripts for automation.
• Hands-on experience with Kafka architecture, including Brokers, Schema Registry, and ZooKeeper.
• Experience managing Kafka components like Cluster linking and Mirror-Making.
• Knowledge of Kafka security protocols such as SASL, JaaS config, Kerberos, and OAuth.
• Ability to develop custom Kafka producers and consumers.
• Experience managing data ingestion from various sources into Kafka topics.

Preferred Skills

• Degree in any field of study.

Key Skills

Education

Any Graduates