Description

You will develop and maintain Hadoop-based applications and data pipelines.

Responsibilities

• Process large datasets using Hadoop ecosystem tools like HDFS, Hive, and Spark.
• Write optimized Hive queries and Spark/MapReduce jobs.
• Perform data ingestion, transformation, and loading (ETL) from multiple data sources.
• Troubleshoot and resolve data processing and performance issues in big data solutions.
• Manage data workflows within Linux/Unix environments.

Required Skills

• 12+ years of experience with the Hadoop ecosystem (HDFS, Hive, Spark, MapReduce).
• Strong SQL and data processing skills.
• Programming proficiency in Python or Scala.
• Experience implementing ETL processes and big data architecture.
• Experience with data ingestion tools such as Kafka.
• Familiarity with Linux/Unix environments.
• Experience with AWS or Azure.

Key Skills

Education

Bachelor's degree