Description

You will design and deploy multi-cloud data pipelines on GCP and Azure, building frameworks for ingestion, processing, and real-time streaming.

This role is hybrid.

Responsibilities

• Develop and deploy data pipelines in GCP and Azure environments.
• Build data ingestion and processing frameworks using Dataflow, Data Composer, and BigQuery.
• Create real-time pipelines using Kafka and Pub/Sub.
• Implement pipeline monitoring, observability, and dashboards with PowerBI or ThoughtSpot.
• Advise on Azure and GCP technology implementations.

Required Skills

• 7+ years developing data pipelines in GCP or Azure.
• 7+ years in one cloud platform and 2-3 years in the other.
• 5+ years with Snowflake, BigQuery, or Databricks.
• 5+ years building frameworks using GCP Dataflow, Data Composer, and BigQuery.
• 4+ years with SQL, Python, Java, and API development.
• 2+ years with real-time pipelines using Kafka and Pub/Sub.
• 2+ years building dashboards with PowerBI and/or ThoughtSpot.
• GCP Professional Architect or GCP Data Engineer certification.
• Bachelor’s or Master’s in computer engineering, computer science, or related field.

Preferred Skills

• Experience with data transformations for retail and e-commerce use cases.
• Knowledge of GitHub Actions for CI/CD.
• Knowledge of building machine learning models.

Key Skills

Education

Bachelor’s or Master’s in computer engineering, computer science