Description

You will engineer and manage large-scale data processing systems, focusing on scalable pipelines and high-performance applications.

This role is on-site.

Responsibilities

• Design and implement scalable data processing pipelines using Hadoop and Spark.
• Develop and maintain high-performance applications in Scala for large dataset analysis.
• Build batch and real-time data processing solutions.
• Optimize Spark jobs and Hadoop workflows for performance and scalability.
• Troubleshoot distributed data processing systems and ensure data integrity.

Required Skills

• 5+ years of experience in data engineering roles.
• Strong proficiency in Scala for complex transformations and performance tuning.
• Hands-on experience with Spark (RDD, DataFrame, Dataset, Spark SQL).
• Working knowledge of Hadoop ecosystem.
• Experience using PySpark and Python.
• Understanding of distributed computing concepts.
• Familiarity with ETL concepts and data warehousing.
• Experience with Git and working in Agile environments.

Key Skills

Education

Any Graduate