Description

You will manage big data infrastructure and develop data processing pipelines within a Hadoop ecosystem.

Responsibilities

• Install, configure, and manage Hadoop Cluster infrastructure and underlying big data components.
• Analyze structured and unstructured datasets using HiveQL, Pig Latin, HBase, and custom Java MapReduce programs.
• Build NIFI templates to ingest real-time data and parse complex JSON objects for Hive storage.
• Develop real-time data processing pipelines using Kafka, Spark, and Scala.
• Automate workflows with Oozie, write shell scripts for data cleansing, and migrate data between HDFS and relational databases using Sqoop.

Required Skills

• Experience with Hadoop or Big Data applications.
• Java programming for MapReduce development.
• Pig Latin for data scripting.
• Kafka for real-time data consumption.
• Shell scripting for data preparation and automation.
• JSON parsing and manipulation.
• Experience with HiveQL, HBase, Spark, and Scala.
• Experience with NIFI, Sqoop, and Oozie.
• Ability to travel or relocate to unanticipated client sites within the USA.

Education and Experience

• Master’s degree in Computer Science, Engineering, IT, or a related field with 6 months of relevant experience.
• OR a Bachelor’s degree in Computer Science, Engineering, IT, or a related field with 5 years of progressive work experience.

Key Skills

Education

Any Graduate