Description

You will build and support database applications to automate data collection, storage, visualization, and transformation. You will migrate existing IBM Streams pipelines to Google Cloud Dataflow and develop custom templates in Apache Beam.

Responsibilities

• Migrate IBM Streams pipelines to Google Cloud Dataflow.
• Build real-time streaming pipelines using Flink and Dataflow for billions of daily records.
• Design and implement automated data collection and storage systems.
• Perform data validation and quality checks across multiple sources.
• Consolidate data from disparate sources for easier access.

Required Skills

• 5+ years of experience as a GCP Data Engineer.
• Hands-on experience with Apache Flink and Apache Beam.
• Proficiency in writing complex SQL queries and stored procedures.
• Experience managing real-time streaming systems at scale.
• Practical experience with Jenkins for CI/CD implementation on the cloud.
• Strong background in database design, data modelling, and data mining.
• Ability to work with large relational databases and multiple data systems.

Key Skills

Education

Any gradudate