Description

Key Skills: Shell Scripting, Pyspark, Linux, Hadoop Ecosystem, UNIX, Python, Big Data

Roles and Responsibilities:

• Administer and maintain Hadoop clusters, ensuring high availability and performance.
• Develop and implement data processing workflows using Pyspark.
• Monitor system performance and troubleshoot issues as they arise.
• Collaborate with data engineers and analysts to optimize data storage and retrieval.
• Write and maintain shell scripts for automation of routine tasks.
• Ensure data security and compliance with organizational policies.

Skills Required:

• Strong expertise in administering Hadoop clusters and related ecosystem components
• Proficiency in Python and Pyspark for developing and optimizing data workflows
• Hands-on experience with Linux/UNIX systems and shell scripting for automation
• Knowledge of Big Data concepts and tools (HDFS, Hive, HBase, Spark)
• Ability to monitor, troubleshoot, and optimize system performance
• Understanding of data security, compliance, and governance best practices

Education: Relevant degree in Computer Science or related field preferred

Key Skills

Education

Any Graduate