Description

Responsibilities

Architect, build, and operate real-time and near-real-time data pipelines
Lead development using Confluent Kafka and Apache Flink
Apache Spark Structured Streaming is acceptable instead of Flink
Design and maintain batch and hybrid ETL pipelines using SQL Server and SSIS
Own data quality, observability, and reliability across pipelines
Define and enforce best practices for:
Streaming semantics
Error handling
Backpressure and performance tuning
Work directly with onsite clients:
Translate requirements into technical solutions
Defend design decisions
Troubleshoot complex production issues
Mentor engineers and raise overall engineering bar
Collaborate with DevOps and platform teams in containerized, distributed environments
 

Required Skills & Experience

8+ years of hands-on data engineering experience
Exceptional SQL skills (complex analytics, performance tuning)
Strong expertise in Python, Java, or .NET (C#)
(at least one OOP language is mandatory)
Deep experience with Confluent Kafka (architecture, partitioning, delivery semantics)
Hands-on streaming experience with:
Apache Flink (preferred) or
Apache Spark Structured Streaming
Strong understanding of:  
ETL / ELT patterns
MPP processing concepts
Data modeling and orchestration
Experience with SQL Server
 

Nice to Have

SSIS experience
Docker / Kubernetes
CI/CD for data pipelines
Cloud or hybrid data platforms
 

Soft Skills (Non-Negotiable)

Exceptional communication and client-facing skills
Ability to lead technical conversations without hand-holding
Strong ownership, accountability, and judgment
Calm under production pressure
 

What Success Looks Like

Streaming systems are scalable, observable, and resilient
Clients trust your technical leadership
Pipelines are built for operability, not heroics
 

Key Skills

Education

Any Graduate