Description

Design and build large-scale data products and distributed processing platforms within the GCP ecosystem.

Responsibilities

• Build and maintain data pipelines using GCP Dataproc, GCS, and BigQuery.
• Develop distributed data processing platforms using Hadoop, Hive, or Spark.
• Design and model schemas for data lakes and RDBMS platforms.
• Manage and process large-scale datasets at multi-TB and PB scales.
• Implement workflow orchestration using Airflow or similar solutions.

Required Skills

• 10+ years of experience developing data warehouse solutions and data products.
• 4+ years of recent experience working with GCP.
• 6+ years of experience with Hadoop, Hive, Spark, or Airflow.
• 5+ years of experience in data lake or RDBMS schema modeling and design.
• Proficiency in Python, Java, or Scala.
• Experience with scripting languages including Perl and Shell.
• Experience working within Scrum/Agile methodologies.
• Exposure to test-driven development and automated testing frameworks.

Preferred Skills

• Any Graduate degree.

Key Skills

Education

Any Gradute