Description

What You'll Do:

• Design and develop scalable and reliable data pipelines for transformation and aggregation of data from multiple sources.
• Collaborate with stakeholders to understand business needs and map them to appropriate data solutions.
• Build and optimize Spark-based applications leveraging Spark Core, Spark SQL.
• Apply performance tuning and optimization techniques such as caching, broadcasting, and efficient data partitioning.
• Work in an Agile/Scrum environment, participating in ceremonies and ensuring sprint deliverables.
• Communicate professionally with end clients via email and phone.

Expertise You'll Bring:

• Strong hands-on experience with Scala and the Spark ecosystem.
• Excellent understanding of Spark architecture and components.
• Willingness and motivation to learn and work with Java as needed.
• Basic knowledge of Linux and shell scripting.
• Experience with Git Bash for version control.
• Familiarity with performance optimization in distributed processing environments

Key Skills

Education

Any Graduate