Description

You will design, build, and maintain data transformation and modernization workflows on the Google Cloud Platform.

Responsibilities

• Build data systems and pipelines using Dataproc, Dataflow, Data Fusion, BigQuery, and Pub/Sub.
• Implement workflows and task scheduling using Cloud Composer and Apache Airflow.
• Develop ETL/ELT pipelines and ingestion processes using Apache PySpark and Dataflow.
• Automate data processing tasks and cloud deployments using Python, Bash, and Terraform.
• Configure IAM roles, service accounts, and access policies to ensure data security.

Required Skills

• 10+ years of relevant experience in data engineering.
• Proficiency in Python and Bash scripting.
• Extensive experience with Apache PySpark.
• Hands-on expertise with GCP services including BigQuery, Cloud Storage, and Dataflow.
• Experience with Cloud Composer and Apache Airflow for orchestration.
• Knowledge of Infrastructure as Code using Terraform or Google Cloud Deployment Manager.
• Experience managing data security through IAM and service accounts.
• Ability to collaborate with Product Owners and Scrum Masters to deliver user stories.

Preferred Skills

• Bachelor's degree in Computer Science or equivalent.

Key Skills

Education

Any Graduate