Description

You will design and implement high-performance data processing applications using the Big Data technology stack.

Responsibilities

• Develop Hadoop MapReduce jobs in Java to process web log data on HDFS.
• Build batch processing applications using Apache Hive, Apache Spark (Scala), and Informatica for Oracle Data Warehouse.
• Write Apache Sqoop scripts to ingest data from RDBMS into Hortonworks Hadoop clusters.
• Develop Spark jobs using DataFrames, Datasets, and Spark SQL to aggregate data.
• Optimize Hive table design via partitioning and bucketing, and automate orchestration with Oozie.

Required Skills

• 3+ years of experience with Apache Spark, Scala, and Java.
• Strong proficiency in Hadoop ecosystem components: Hive, Sqoop, HDFS, MapReduce, and Oozie.
• Experience with Kafka and Kerberos security configurations.
• Proficiency in Oracle SQL, including query performance tuning and index optimization.
• UNIX Shell Scripting and Informatica BI.
• Experience with testing against engines like Druid, Apache Phoenix, or Presto.
• Any graduate degree.

Key Skills

Education

Any graduate