Description

You will design, build, and optimize data pipelines for batch and real-time processing while leading engineering teams and scrum processes.

This role is on-site.

Responsibilities

• Design and implement optimized data pipelines for batch and real-time processing.
• Analyze, test, and implement system applications and processing flows.
• Lead engineering and scrum teams, managing delivery and technical direction.
• Document technical specifications, functional requirements, and system architecture.
• Optimize complex SQL queries and data workflows for large-scale distributed systems.

Required Skills

• 7+ years of software development experience.
• 3+ years of hands-on experience with MapReduce, Hive, and Spark (Core, SQL, PySpark).
• Deep expertise in Hadoop and Spark architecture and working principles.
• Proficiency in Core Java, Python, or Scala.
• Experience writing and optimizing complex SQL (Hive/PySpark-dataframes) for large-scale data processing.
• Strong understanding of distributed ecosystems and data warehousing concepts.
• Hands-on experience with UNIX shell scripting.
• Bachelor's degree in Engineering, Computer Science, or equivalent.

Preferred Skills

• Experience designing and building solutions using Kafka streams or queues.
• Knowledge of cloud platforms (GCP/AWS) and building microservices.

Key Skills

Education

Any Gradutate