Description

You will design, implement, and optimize data processing pipelines within Hadoop and HDFS environments.

Responsibilities

• Build and maintain scalable data applications using PySpark and Python.
• Design efficient data transformations and analytics solutions for large-scale datasets.
• Ensure reliability, scalability, and performance of distributed data processing pipelines.
• Troubleshoot and debug issues across Hadoop, HDFS, and PySpark environments.
• Collaborate with data engineers and scientists to translate business requirements into technical solutions.

Required Skills

• 11+ years of experience in big data engineering.
• Proficiency in Python and PySpark programming.
• Hands-on experience with Hadoop and HDFS.
• Strong knowledge of Spark, MapReduce, and Hive.
• Solid understanding of distributed computing and parallel processing concepts.
• Proficiency in SQL and working with relational databases.
• Experience with data serialization formats such as Avro, Parquet, or ORC.
• Any Graduate degree.

Key Skills

Education

Any Graduate