Description

You will design and manage data engineering workflows within the Google Cloud Platform environment.

Responsibilities

• Develop and maintain infrastructure using Terraform.
• Build and optimize data pipelines using Apache Spark, PySpark, Hadoop, and Hive.
• Design and implement scalable database models and data structures.
• Manage cloud-based services including BigQuery, Cloud Run, and Google Cloud Storage.
• Utilize GCP APIs and SDKs to automate data engineering tasks.

Required Skills

• 5+ years of experience in data engineering or cloud environments.
• Deep expertise in Google Cloud Platform (GCP).
• Proficiency with Terraform for infrastructure development.
• Strong experience with Apache Spark and PySpark.
• Hands-on experience with Hadoop and Hive.
• Advanced SQL skills for data manipulation and analysis.
• Practical knowledge of database modeling and design.
• Familiarity with cloud computing fundamentals including VMs, storage, and networking.
• Experience with BigQuery and Cloud Run.

Preferred Skills

• Experience writing Python scripts for data engineering using NumPy, Pandas, or SciPy.
• Background in the healthcare or insurance industries.

Key Skills

Education

Any Graduate