Description

Architect and operate real-time and near-real-time data pipelines for onsite clients.

This role is hybrid.

Responsibilities

• Build and maintain streaming pipelines using Confluent Kafka and Apache Flink (or Spark Structured Streaming).
• Design batch and hybrid ETL pipelines using SQL Server and SSIS.
• Define and enforce best practices for streaming semantics, error handling, backpressure, and performance tuning.
• Ensure data quality, observability, and reliability across all pipeline components.
• Mentor engineers, troubleshoot complex production issues, and translate client requirements into technical solutions.

Required Skills

• 8+ years of hands-on data engineering experience.
• Exceptional SQL skills, including complex analytics and performance tuning.
• Strong expertise in at least one OOP language: Python, Java, or .NET (C#).
• Deep experience with Confluent Kafka, including architecture, partitioning, and delivery semantics.
• Hands-on streaming experience with Apache Flink or Apache Spark Structured Streaming.
• Strong understanding of ETL/ELT patterns, MPP processing, and data modeling.
• Experience with SQL Server.

Preferred Skills

• SSIS experience.
• Experience with Docker, Kubernetes, and CI/CD for data pipelines.
• Familiarity with cloud or hybrid data platforms.

Key Skills

Education

Any Graduate