Description

You will build and operate real-time and near-real-time Kafka-based data pipelines.

Responsibilities

• Architect, build, and operate real-time and near-real-time Kafka-based pipelines, focusing on Confluent Kafka.
• Design and implement stream processing using Apache Flink or Apache Spark Structured Streaming.
• Define Kafka best practices covering topic design, partitioning, delivery semantics, error handling, and backpressure management.
• Design and maintain batch and hybrid ETL pipelines using SQL Server ETL tools.
• Own data quality, reliability, and observability across all streaming pipelines.

Required Skills

• 8+ years of hands-on data engineering experience.
• Deep expertise in Confluent Kafka: Architecture, cluster design, topic/partition strategy, and delivery semantics.
• Strong hands-on experience with Apache Flink or Apache Spark Structured Streaming.
• Excellent SQL skills for complex analytics and performance tuning.
• Strong experience in Python, Java, or .NET (C#); proficiency in at least one OOP language is mandatory.
• Solid understanding of streaming and event-driven architectures and ETL/ELT patterns.
• Experience with SQL Server and designing data models.

Key Skills

Education

Bachelor's degree