Description

You will design and implement scalable data processing pipelines and maintain Java applications for large-scale data ingestion and transformation.

Responsibilities

• Design and implement scalable data processing pipelines using Apache Kafka, Spark, and Structured Streaming.
• Develop and maintain Java applications for data ingestion, transformation, and storage.
• Integrate data processing solutions with AWS services including Amazon MSK, S3, Lambda, and EMR.
• Implement real-time data processing solutions to handle high volumes of data efficiently.
• Optimize pipelines for performance and scalability while monitoring Kafka and Spark applications for issues.

Required Skills

• 5+ years of industry experience as a Java developer.
• Expertise in Java, Kafka, Spark, and Structured Streaming.
• Proficiency with Spark SQL.
• Hands-on experience with AWS services and cloud-based architectures.
• Experience in system design, application development, testing, and operational stability.
• Proven ability to develop, debug, and maintain code within a large corporate environment.
• Practical experience in designing real-time data processing, enrichment, and transformation solutions.
• Any Graduate degree.

Key Skills

Education

Any Graduate