Description

You will build and manage data pipelines within the Google Cloud ecosystem.

Responsibilities

• Develop and implement data pipelines using Google Cloud Dataflow.
• Optimize data processing workflows for performance and cost efficiency.
• Collaborate with IBM UDMH Data Modelers and analysts to define data requirements.
• Ensure data quality and integrity throughout the entire pipeline lifecycle.
• Manage administrative tasks including project setup, user access, and job scheduling.

Required Skills

• 10+ years of IT experience with deep expertise in BigQuery and SQL.
• Strong proficiency in Python and Java for Google Cloud Dataflow.
• Hands-on experience with ETL processes and data migration.
• Mastery of Google Cloud Platform services: Cloud Dataflow, Cloud Pub/Sub, Cloud Storage, and Google Composer.
• Experience with BigQuery, Cloud Functions, Airflow, and SQL.
• Solid understanding of Git workflows using GitHub, GitLab, or Bitbucket.
• Experience working in Agile environments and CI/CD pipelines.
• Ability to write and test scripts using Unix Shell and Oracle PL/SQL.
• Experience conducting unit testing and data validation.

Key Skills

Education

Any Gradute