Description

You will design and maintain HDFS-based solutions, focusing on data reporting, ETL automation, and cluster performance tuning.

Responsibilities

• Build Sqoop jobs to import and export data within the Hadoop ecosystem.
• Develop automation scripts using Shell and Python to streamline workflows.
• Set up Hive structures, manage environment migrations, and troubleshoot issues across Hive, Impala, Spark, and Sqoop.
• Monitor Hadoop cluster job performance and perform end-to-end tuning for MapReduce routines against large datasets.
• Manage HDFS file systems, review Linux and Hadoop log files, and collaborate with infrastructure and analytics teams to ensure data quality.

Required Skills

• 5+ years of experience in Hadoop development.
• Proficiency with Hive, HBase, Spark, and MapReduce.
• Strong SQL and RDBMS utility skills.
• Hands-on experience with Sqoop and HDFS.
• Experience with Shell scripting and Python for automation.
• Competency in using Eclipse, XML, and JSON.
• Ability to manage and monitor Linux log files.

Key Skills

Education

Any Graduate