Description

You will build and maintain high-volume data pipelines using GCP services.

Responsibilities

• Design and build data transformation and processing solutions on GCP.
• Build high-volume data pipelines supporting large-scale search applications.
• Analyze and manipulate data sets from disparate sources using various tools.
• Design and optimize queries against data within HDFS environments.
• Collaborate with internal and cross-functional teams.

Required Skills

• 5+ years of experience in data engineering.
• Expertise in Python and PySpark.
• Strong experience with Apache Spark and BigQuery.
• Proficiency with GCP services including Cloud Composer, Cloud Dataflow, and Cloud DataProc.
• Hands-on experience with Cloud SQL and Hadoop architecture/HDFS commands.
• Familiarity with REST APIs and Apache NiFi.
• Knowledge of Hive.

Key Skills

Education

Any Graduate