Description

You will develop and manage big data pipelines using the Hadoop ecosystem.

Responsibilities

• Develop data ingestion processes from various platforms using batch and real-time modes.
• Analyze data using HiveQL, Pig Latin, and custom MapReduce programs.
• Model and transform data, designing integration processes from raw to curated data warehouse zones.
• Install and configure Hadoop ecosystem components like Flume-ng, HBase, ZooKeeper, and Spark.
• Utilize Spark for data processing, including SQL and Streaming applications.

Required Skills

• 5+ years of experience with Hadoop Framework and its ecosystem.
• Proficiency in HDFS, MapReduce, Hive, Pig, Sqoop, HBase, ZooKeeper, Oozie, Spark, and Scala.
• Strong knowledge of Hadoop Architecture, including Name Node, Data Node, and MapReduce programming.
• 4+ years experience in data modeling and data transformation.
• Experience with data processing frameworks, specifically Apache Spark.
• Experience with cloud technologies (AWS, GCP, or Azure) focused on big data stacks.
• Familiarity with data ingestion patterns (batch and real-time).

Key Skills

Education

Any Graduate