Description

You will design, implement, and manage Hadoop infrastructure and the broader data ecosystem.

Responsibilities

• Design, implement, and maintain large-volume Hadoop clusters including HDFS, YARN, and MapReduce.
• Administer and monitor clusters to ensure high availability, reliability, and performance.
• Troubleshoot issues related to infrastructure, data ingestion, processing, and storage.
• Implement security measures including authentication, authorization, encryption, and RBAC.
• Collaborate with DevOps to automate infrastructure provisioning, deployment, and management.

Required Skills

• 3 years of experience in Hadoop platform engineering.
• Expertise in managing HDFS, YARN, and MapReduce components.
• Experience with cluster administration, monitoring, and performance tuning.
• Hands-on experience implementing security protocols, authentication, and encryption.
• Knowledge of data ingestion, processing, and storage troubleshooting.
• Ability to automate infrastructure provisioning and deployment processes.
• Experience with backup and disaster recovery planning.
• Competency in managing resource utilization and job scheduling/prioritization.

Key Skills

Education

Not specified