Description

You will design and develop scalable data pipelines for transforming and aggregating data from multiple sources.

This role is on-site.

Responsibilities

• Build and optimize Spark-based applications using Spark Core and Spark SQL.
• Apply performance tuning techniques such as caching, broadcasting, and efficient data partitioning.
• Collaborate with stakeholders to map business needs to data solutions.
• Participate in Agile ceremonies and ensure sprint deliverables.
• Communicate professionally with end clients via email and phone.

Required Skills

• Strong hands-on experience with Scala and the Spark ecosystem.
• 10+ years of experience in data engineering.
• Excellent understanding of Spark architecture and components.
• Ability to work with Java as needed.
• Basic knowledge of Linux and shell scripting.
• Experience with Git Bash for version control.
• Familiarity with performance optimization in distributed processing environments.

Key Skills

Education

Any Graduate