Description

You will develop and deploy Java-based Apache Spark applications for large-scale distributed data processing.

Responsibilities

• Develop, test, and deploy Java-based Apache Spark applications and microservices.
• Optimize Spark jobs for performance, scalability, and reliability.
• Design and implement data storage and retrieval solutions using Cloudera Hadoop components.
• Build and manage ETL pipelines for real-time and batch data ingestion using Kafka, NiFi, and Sqoop.
• Troubleshoot performance bottlenecks across Spark and Cloudera platforms.

Required Skills

• 8+ years of experience in application development with a focus on Java and Big Data.
• Strong hands-on experience with Java, Apache Spark, and Spark SQL.
• Proficiency in Cloudera Hadoop (CDH) components including HDFS, Hive, Impala, HBase, Kafka, and Sqoop.
• Experience with SQL and NoSQL databases such as PostgreSQL, HBase, and Hive.
• Knowledge of data warehousing, dimensional modeling, and data lakes.
• Experience with real-time data streaming using Kafka, Flume, Oozie, and NiFi.
• Competency with Git, Bitbucket, and CI/CD pipelines using Jenkins or GitLab.

Preferred Skills

• Experience deploying Spark applications in both cloud and on-premises Cloudera environments.

Key Skills

Education

Any Graduate