Description

You will build and maintain real-time streaming and data processing pipelines.

Responsibilities

• Develop clean, performant code using Scala for data processing tasks.
• Build and manage real-time streaming pipelines using Apache Spark and Structured Streaming.
• Implement messaging and streaming solutions using Apache Kafka.
• Design and execute data modeling, ETL processes, and pipeline orchestration.
• Operate within the Hadoop ecosystem using tools like Hive and Parquet.

Required Skills

• 5+ years of experience with Apache Spark and Structured Streaming.
• Strong proficiency in Scala.
• 3+ years of experience with Apache Kafka.
• Experience with Data Modeling and ETL workflows.
• Knowledge of Hive and Parquet.
• Experience with AWS (EMR, MSK), Azure, or GCP.
• Any Graduate degree.

Preferred Skills

• Experience with CI/CD, Docker, and Kubernetes.

Key Skills

Education

Any Gradute