Description

You will build and maintain data pipelines and processing systems.

Responsibilities

• Develop and implement solutions using Hadoop, Spark, and Hive.
• Write and optimize code in Scala, Java, or Python.
• Perform data manipulation using SQL, HQL, and SparkSQL.
• Manage data import/export tasks using Sqoop.
• Conduct data wrangling to structure data for insights and collaborate in an Agile environment.

Required Skills

• 3+ years experience as a Big Data Software Engineer.
• Proficiency in Scala or expert-level in Java.
• Strong knowledge of Hadoop Distributed File System (HDFS) and Spark commands.
• 3+ years experience with SQL and various RDBMS.
• Familiarity with Hive and HQL.
• 3+ years strong command of Unix/Linux and shell scripting.
• Proficiency in GitHub.
• Basic experience with job scheduling tools like AutoSys.

Preferred Skills

• Experience with Kafka or Spark Streaming.
• Knowledge of Scala API for Hadoop.

Key Skills

Education

Bachelor's degree