Description

You will own the design and management of data processing workflows within a cloud environment, focusing on GCP infrastructure.

Responsibilities

• Build and maintain scalable data pipelines using Spark and Scala.
• Orchestrate complex workflows and task scheduling using Airflow.
• Develop and optimize SQL queries for data warehousing requirements.
• Implement CI/CD practices to automate data engineering deployments.
• Manage large-scale data processing tasks within Hadoop ecosystems.

Required Skills

• 10+ years of professional experience in data engineering.
• Proficiency with Spark and Scala for distributed computing.
• Strong experience with Hadoop and distributed systems.
• Expertise in SQL and data warehousing principles.
• Hands-on experience with Airflow for workflow orchestration.
• Practical knowledge of Cloud infrastructure, specifically GCP.
• Experience implementing CI/CD pipelines.

Preferred Skills

• Experience with Kafka and streaming architectures.
• Hands-on experience with Looker.

Key Skills

Education

Any Graduate