Description

You will design and build data pipelines using cloud services and engineering best practices.

Responsibilities

• Develop and maintain data processing workflows using GCP services.
• Implement infrastructure as code using Terraform.
• Build and manage data ingestion and export jobs using scripting.
• Operate and maintain orchestration pipelines using Airflow.

Required Skills

• 4+ years of strong experience in Python and PySpark development.
• 4+ years of experience with shell scripting for data jobs.
• 2+ Years of experience with GCP services including BigQuery, Dataflow, DataProc, DataFusion, and Cloud Storage.
• 2+ Years of experience with Airflow and Kafka.
• 2+ Years of experience with Git or similar version control.
• 2+ Years of experience with CI/CD tooling like Tekton.
• Experience with SQL on Cloud SQL and Postgres.
• Knowledge of API development.

Key Skills

Education

Bachelor's degree