Description

Design and maintain scalable data pipelines on Google Cloud Platform to ingest, transform, and store data for analytics.

This role is hybrid.

Responsibilities

• Build and optimize complex data pipelines using Dataflow and Apache Airflow to move data from Google Cloud Storage to BigQuery.
• Develop event-driven architectures using Pub/Sub for real-time and batch processing, including trigger and message handling.
• Implement API-integrated workflows to fetch external data, process responses, and load results into downstream systems.
• Manage BigQuery DDLs, views, and authorized views, ensuring secure data access through appropriate roles and permissions.
• Improve ETL pipeline and SQL query performance, reducing processing time and enhancing warehouse efficiency.

Required Skills

• 7+ years of experience in data engineering.
• Strong proficiency in Python and Java.
• Expertise in Google Cloud Platform services, specifically BigQuery and Dataflow.
• Hands-on experience with Apache Airflow for workflow orchestration.
• Advanced SQL skills for complex queries and data modeling.
• Experience with SQL-based report generation and automated distribution.
• Ability to create interactive dashboards using Looker Studio.
• Bachelor's degree in a relevant field.

Preferred Skills

• Experience with DEV and UAT testing for data quality validation.
• Proven track record of collaborating with stakeholders to gather requirements and deliver data solutions.

Key Skills

Education

Bachelor's degree