Description

You will design, develop, and maintain Hadoop applications, handling data ingestion, processing, and storage.

Responsibilities

• Design and develop Hadoop applications for data ingestion, processing, and storage.
• Optimize Hadoop MapReduce jobs, Hive queries, and Pig scripts for analysis.
• Manage Hadoop cluster configuration, monitoring, and troubleshooting.
• Collaborate with cross-functional teams to define requirements and ensure data quality.

Required Skills

• 10+ years of experience in big data development.
• Strong programming skills in Java, Python, or Scala.
• In-depth knowledge of Hadoop, HDFS, MapReduce, Hive, Pig, HBase, and Spark.
• Experience with big data technologies including NoSQL databases, data warehousing, and ETL tools.
• Experience with data processing frameworks such as Hadoop, Spark, and Flink.
• Conceptual understanding of multithreading and distributed computing concepts in PySpark.

Preferred Skills

• Experience with cloud-based big data platforms like AWS EMR, Azure HDInsight, or Google Cloud Dataproc.
• Familiarity with machine learning frameworks such as TensorFlow, PyTorch, or Scikit-learn.

Key Skills

Education

Bachelor's degree