Description

Design and manage end-to-end data engineering lifecycles and solution architectures on Google Cloud Platform.

Responsibilities

• Design and develop ETL/ELT frameworks using BigQuery, including nested queries, clustering, and partitioning.
• Build and manage data pipelines using Google Cloud data products.
• Automate data loads from BigQuery through APIs or scripting.
• Own end-to-end solution design, including prototyping, usability testing, and data visualization.
• Manage data integration, transformation, quality, and lineage processes.

Required Skills

• 5+ years of professional experience in data engineering.
• Deep expertise in Google Cloud Platform (GCP) and BigQuery (GBQ).
• Advanced Python programming skills.
• Hands-on experience with Google data products: Dataflow, Dataproc, Dataprep, and Cloud Composer.
• Proficiency with Airflow for creating DAGs, configuring variables, and scheduling.
• Experience with PySpark and Pandas.
• Knowledge of Big Data technologies including Spark, Hadoop, Hive, and MapReduce.
• Experience using DBT to create lineage in GCP.
• Proficiency in SQL and NoSQL modern data stores.
• Experience with scripting languages including YAML.

Preferred Skills

• Experience working in a DevSecOps (CI/CD) environment.

Key Skills

Education

Any Graduate