Description

You will design and maintain large-scale data pipelines using the Hadoop ecosystem. You own the development of distributed systems and real-time streaming solutions.

Responsibilities

• Build and optimize batch processing workflows using Oozie, Pig, and Hive.
• Develop high-throughput streaming applications with Apache Spark, Flink, and Storm.
• Implement data ingestion and queuing systems using Kafka.
• Manage HDFS storage and ensure high availability for distributed components.
• Write robust Java code for MapReduce jobs and core ecosystem integrations.

Required Skills

• 10+ years of professional Java development experience.
• 5+ years of hands-on experience with the Hadoop ecosystem.
• Strong proficiency in Hive, Pig, Oozie, and Spark.
• Experience with real-time streaming technologies such as Apache Storm or Flink.
• Practical knowledge of Kafka or similar message queuing systems.
• Solid understanding of HDFS, MapReduce, and distributed system architecture.
• Ability to design for high availability and scalability.

Key Skills

Education

Any Graduate