Description

You will design, develop, and optimize data pipelines for large-scale analytics using GCP.

Responsibilities

• Implement and manage ETL processes leveraging GCP services like BigQuery Dataflow and Airflow.
• Develop scalable, maintainable, and reusable data pipelines for BI and analytics.
• Optimize SQL queries and data models for cost efficiency in BigQuery.
• Integrate Hadoop ecosystem components with GCP services to enhance data processing.
• Automate workflow orchestration using Apache Airflow for data operations.

Required Skills

• 5+ years of professional experience.
• Strong proficiency in Python and Scala.
• Experience with GCP BigQuery and GCP DataProc.
• Working knowledge of Apache Airflow and GCP Cloud Composer.
• Familiarity with GCP Storage and GCP DataStream.
• Experience integrating Hadoop ecosystem components.
• Ability to write and optimize complex SQL.

Key Skills

Education

Any Degree Graduation