Description

You will develop big data solutions using Scala, Spark, and Hadoop to build and maintain data processing pipelines.

This role is on-site.

Responsibilities

• Develop and implement data processing pipelines using Spark Core, Spark SQL, and Spark Streaming.
• Manage and interact with the Hadoop ecosystem, including HDFS, Hive, Oozie, and Sqoop.
• Write efficient SQL queries to handle large datasets.
• Automate processes using shell scripting.

Required Skills

• 6+ years of professional experience.
• Expertise in Scala or Python.
• Hands-on experience with Spark architecture, Spark Core, Spark SQL, and Spark Streaming.
• Proficiency in the Hadoop ecosystem (HDFS, Hive, Oozie, Sqoop).
• Strong SQL and coding abilities.
• Proficiency in shell scripting for automation.
• Experience working within an Agile framework.

Preferred Skills

• Familiarity with Kafka.

Key Skills

Education

Any Graduate