Description

You will design and develop large-scale data processing applications using Java and Apache Spark.

Responsibilities

• Develop, test, and deploy Java-based Apache Spark applications for large-scale data processing.
• Optimize and fine-tune Spark jobs for performance, scalability, and reliability.
• Implement Java-based microservices and APIs for data integration.
• Design and implement high-performance data storage and retrieval solutions using Cloudera Hadoop components.
• Troubleshoot and resolve performance bottlenecks across Spark and Cloudera platforms.
• Manage data ingestion using Kafka, Flume, Oozie, and NiFi for real-time and batch workflows.
• Maintain version control and CI/CD pipelines for Spark applications.

Required Skills

• 9+ years of experience in application development with a focus on Java and Big Data.
• Strong hands-on experience with Java, Apache Spark, and Spark SQL.
• Proficiency in Cloudera Hadoop (CDH) components including HDFS, Hive, Impala, HBase, Kafka, and Sqoop.
• Experience building and optimizing ETL pipelines for large-scale workloads.
• Practical knowledge of SQL and NoSQL databases such as HBase, Hive, and PostgreSQL.
• Deep understanding of data warehousing, dimensional modeling, and data lakes.
• Experience with version control (Git, Bitbucket) and CI/CD tools (Jenkins, GitLab).
• Familiarity with real-time streaming technologies including Kafka, Flume, Oozie, and NiFi.

Key Skills

Education

Any Gradute