Description

You will build and maintain data pipelines using the Hadoop ecosystem, focusing on real-time and stream processing systems. You will provide production support and ensure system stability across various Hadoop distributions.

Responsibilities

• Develop and maintain Hadoop-based data solutions using Spark, Oozie, HDFS, Yarn, and Hive.
• Implement real-time and stream processing architectures to handle high-volume data flows.
• Provide production support, troubleshooting, and maintenance for distributed systems.
• Deploy and manage Hadoop clusters across different distributions including CDH, CDP, and MapR.
• Automate workflows and scripting tasks using Python.

Required Skills

• 5+ years of development experience with the Hadoop ecosystem.
• Strong proficiency in Java for core development tasks.
• Hands-on experience with Apache Spark, Oozie, HDFS, Yarn, and Hive.
• Knowledge of multiple Hadoop distributions (CDH, CDP, MapR).
• Experience supporting real-time and stream processing systems.

Preferred Skills

• Proficiency in Python scripting.
• Experience with AWS cloud services.
• Familiarity with Apache Airflow for workflow orchestration.

Key Skills

Education

Any Graduate