Description

You will design, build, and maintain large-scale data pipelines and warehouse solutions on Google Cloud Platform. You will own the data architecture, from logical modeling to physical implementation, ensuring scalability and performance.

Responsibilities

• Design and develop big data applications and data products using GCP services like Dataproc, GCS, and BigQuery.
• Build and automate data pipelines using Apache Spark, Apache Hive, Apache Kafka, and Apache Airflow.
• Maintain existing systems, participate in rotational on-call support, and drive ongoing enhancements.
• Mentor junior engineers, lead design reviews, and groom/prioritize the backlog in JIRA.
• Collaborate with the team to share domain knowledge and drive technical decisions.

Required Skills

• 12+ years total IT experience with 5+ years of recent, hands-on GCP experience.
• Proven experience building data pipelines and data warehouse solutions on GCP.
• 5+ years of hands-on experience with distributed data processing using Hadoop, Hive, or Spark.
• Strong proficiency in Python, Java, or Scala for data engineering tasks.
• Experience with workflow orchestration, specifically Apache Airflow.
• Ability to model and design schemas for data lakes or RDBMS platforms.
• Experience processing multi-TB/PB scale datasets.

Preferred Skills

• Experience with Gitflow, Atlassian products (BitBucket, Confluence), and CI tools (Bamboo, Jenkins).
• Exposure to test-driven development and automated testing frameworks.
• Background in Scrum/Agile methodologies and working in offshore/managed outcome models.

Key Skills

Education

Any Gradute