Description

You will design and build scalable, low-latency, fault-tolerant streaming data platforms and pipelines.

Responsibilities

• Build and maintain scalable, repeatable, and secure data pipelines serving multiple internal users.
• Design distributed streaming data pipelines and analytics data stores using streaming frameworks.
• Develop platforms for gathering, storing, and performing real-time processing on data for decision-making systems.
• Identify and implement improvements for data reliability, efficiency, and quality.
• Collaborate with business and technology stakeholders to drive adoption of the Data Analytics platform.

Required Skills

• 5+ years of production experience building fault-tolerant data pipelines that clean, transform, and aggregate messy data.
• Hands-on experience with distributed stream processing frameworks such as Kafka, Spark Streaming, or Storm.
• Proficiency with Kafka, Spark, SQL Server, and Oracle.
• Experience with micro-services and deployment platforms like Kubernetes.
• Strong knowledge of data modeling, data structures, and their limitations in specific use cases.
• Experience with object-oriented design, coding, and testing patterns.
• Practical experience in performance tuning, optimization, and bottleneck analysis.
• Background in engineering commercial or open-source software platforms and large-scale data infrastructures.
• Experience working with agile or rapid application development methods.

Preferred Skills

• Knowledge of data warehousing principles, schema design, data governance, and database security.

Key Skills

Education

Any Gradute