Description

You will design and execute migration strategies from Redshift to BigQuery, ensuring seamless data synchronization. You own the architecture of large-scale data solutions within the GCP ecosystem.

Responsibilities

• Design custom solutions for large-scale data migration and synchronization between Redshift and BigQuery.
• Develop streaming pipelines and custom templates using Dataflow and Beam.
• Build and troubleshoot complex data pipelines within the GCP ecosystem.
• Develop custom plugins for Data Fusion to extend integration capabilities.
• Manage and optimize data processing workloads on Cloud Dataproc.

Required Skills

• 10+ years of total IT experience.
• 5+ years of experience with Data Fusion and Dataflow using Beam programming in Java.
• GCP Data Engineering certification is mandatory.
• Expertise in PySpark programming.
• Extensive experience with GCP services including BigQuery, Cloud Composer, Cloud Storage, Pub/Sub, and IAM.
• Hands-on experience with Cloud Dataproc.
• Proven track record in architecting Redshift to BigQuery migrations.

Preferred Skills

• Knowledge of SAP BODS or Talend data pipelines.

Key Skills

Education

Any Graduate