Description

You will design, develop, and optimize large-scale data processing applications and ETL pipelines using Java and Apache Spark.

This role is on-site.

Responsibilities

• Develop and deploy Java-based Apache Spark applications for large-scale data processing.
• Optimize and fine-tune Spark jobs for performance, scalability, and reliability.
• Implement Java-based microservices and APIs for data integration.
• Troubleshoot and resolve performance bottlenecks within Spark and Cloudera platforms.
• Build and maintain ETL pipelines and real-time data ingestion workflows using Kafka, Flume, Oozie, and NiFi.

Required Skills

• 8+ years of application development experience with a focus on Java and Big Data.
• Strong hands-on experience with Java, Apache Spark, and Spark SQL.
• Proficiency with Cloudera Hadoop (CDH) components: HDFS, Hive, Impala, HBase, Kafka, and Sqoop.
• Experience with SQL and NoSQL databases such as HBase, Hive, and PostgreSQL.
• Knowledge of data warehousing, dimensional modeling, and data lakes.
• Experience with version control (Git, Bitbucket) and CI/CD pipelines (Jenkins, GitLab).
• Practical experience with real-time data streaming using Kafka, Flume, Oozie, and NiFi.
• Ability to optimize Spark applications for high performance.

Preferred Skills

• Experience with real-time data streaming using Kafka, Flume, Oozie, and NiFi.
• Ability to optimize Spark applications for high performance.

Key Skills

Education

Any Gradute