Description

You will engineer and optimize data processing pipelines within a GCP environment, ensuring reliable streaming and batch processing.

This role is hybrid.

Responsibilities

• Engineer data pipelines using Scala and Spark for streaming and batch workloads.
• Build and maintain CI/CD workflows using GitLab pipelines.
• Troubleshoot Spark jobs and resolve performance issues using the Spark UI.
• Deploy and manage data infrastructure on GCP.

Required Skills

• 5+ years of professional experience in data engineering.
• Strong proficiency in Scala and Spark (batch and streaming).
• Experience with PySpark.
• Hands-on experience with GitLab pipelines and CI/CD concepts.
• Practical knowledge of GCP.
• Ability to troubleshoot complex Spark jobs via Spark UI.
• Immediate ability to contribute to production environments without training.

Preferred Skills

• Any graduate degree.

Key Skills

Education

Any Graduate