Description

You will design, develop, and maintain scalable data processing solutions using Apache Spark and Scala.

Responsibilities

• Write and optimize Spark jobs for performance and scalability in both batch and real-time frameworks.
• Build and manage data pipelines using Databricks, Snowflake, and Kafka.
• Develop and optimize SQL queries for data extraction, transformation, and loading.
• Produce unit tests for Spark transformations and helper methods to ensure data quality and integrity.
• Monitor and troubleshoot data processing issues.

Required Skills

• 6+ years of experience designing and maintaining data processing solutions with Apache Spark (Scala).
• Hands-on experience with Databricks and on-prem Hadoop clusters.
• Proficiency with Snowflake and Kafka for real-time data streaming and integration.
• Strong command of SQL and Unix environments.
• Deep understanding of functional programming and RESTful APIs.
• Experience implementing both batch and real-time data processing frameworks.
• Any Graduate degree.

Key Skills

Education

Any Graduate