Description

You will build and maintain large-scale data processing systems using Apache Spark and Scala.

Responsibilities

• Build and maintain data processing pipelines using Apache Spark, including Spark SQL, Spark Streaming, and Spark MLlib.
• Design and implement ETL processes, data warehousing solutions, and data models.
• Apply distributed computing principles to manage cluster operations and data workloads.
• Collaborate with technical and non-technical stakeholders to define and implement data requirements.

Required Skills

• 10+ years of experience as a Spark Developer or in a similar big data role.
• Strong proficiency in Apache Spark (SQL, Streaming, and MLlib).
• Expertise in Scala or Python for Spark development.
• Deep understanding of ETL, data warehousing, and data modeling.
• Solid grasp of distributed computing and cluster management.
• Bachelor's degree in Computer Science, Engineering, or a related field.
• Experience with Git and agile workflows.

Preferred Skills

• Experience with cloud platforms including AWS, Azure, or GCP.
• Familiarity with containerization using Docker and Kubernetes.

Key Skills

Education

Bachelor's degree