Description

Manage and optimize Cloudera Data Platform (CDP) and CDH environments, including Apache Spark, Kafka, and Hadoop ecosystems.

This role is on-site.

Responsibilities

• Install, configure, and manage Cloudera Manager, HDFS, YARN, Hive, and HBase components.
• Deploy and maintain Spark clusters for batch and real-time data processing, optimizing jobs via memory management and execution tuning.
• Install and configure Apache Kafka clusters, ensuring proper topics, partitions, and replication factors for fault tolerance and data consistency.
• Monitor cluster health and performance using Cloudera Manager, troubleshooting lag, throughput issues, and consumer-related problems.
• Perform cluster upgrades, patch management, and capacity planning to maintain optimal system performance.

Required Skills

• 5+ years of experience with Cloudera, Apache Spark, Apache Kafka, Hadoop, HDFS, YARN, Hive, and HBase.
• Proficiency in Cloudera Manager and CDP (Cloudera Data Platform) administration.
• Experience with cluster upgrades, patch management, and capacity planning.
• Skills in performance tuning, troubleshooting data processing issues, and implementing fault tolerance.
• Ability to monitor tools and manage data consistency management.

Preferred Skills

• Experience with advanced performance tuning and deep troubleshooting of Kafka and Spark issues.
• Familiarity with monitoring tools for comprehensive cluster visibility.

Key Skills

Education

Any Graduate