Description

Responsibilities:

• Design and Development: Design, develop, and maintain Hadoop applications, including data ingestion, processing, and storage.

• Data Processing: Develop and optimize Hadoop MapReduce jobs, Hive queries, and Pig scripts for data processing and analysis.

• Data Management: Manage and maintain Hadoop clusters, including configuration, monitoring, and troubleshooting.

• Collaboration: Work with cross-functional teams to identify and prioritize project requirements, provide technical guidance, and ensure data quality.

• Innovation: Stay up to date with industry trends, new technologies, and best practices in big data and Hadoop.

Required Skills:

• Hadoop Ecosystem: In-depth knowledge of Hadoop, HDFS, MapReduce, Hive, Pig, HBase, and Spark.

• Programming: Strong programming skills in Java, Python, or Scala.

• Big Data Technologies: Experience with big data technologies, including NoSQL databases, data warehousing, and ETL tools.

• Data Processing: Experience with data processing frameworks, including Hadoop, Spark, and Flink.

• Very good conceptual understanding of Multithreading, distributed computing concepts of Pyspark

• Communication: Excellent communication and collaboration skills.

Preferred Skills:

• Cloud Experience: Experience with cloud-based big data platforms, such as AWS EMR, Azure HDInsight, or Google Cloud Dataproc.

• Machine Learning: Experience with machine learning and deep learning frameworks, such as TensorFlow, PyTorch, or Scikit-learn.

• Certifications: Relevant certifications, such as Cloudera Certified Professional (CCP) or Hortonworks Certified Developer (HCD)

Key Skills

Education

Bachelor's degree