Description

You will design, develop, and maintain data architectures and orchestration pipelines within a GCP environment.

Responsibilities

• Design and implement end-to-end ETL/ELT pipelines using Python.
• Build and manage data orchestration workflows using Apache Airflow and Apache Beam.
• Write and optimize complex SQL code for data processing.
• Develop and maintain database designs and technical architectures.
• Implement LLM and Prompt Engineering solutions within data workflows.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python for data pipeline development.
• Expertise in writing and optimizing SQL.
• Hands-on experience with Google Cloud Platform (GCP).
• Experience with BigQuery.
• Experience with Apache Beam (Cloud Dataflow).
• Experience with Apache Airflow (Cloud Composer).
• Solid understanding of ETL/ELT design and database design.
• Practical knowledge of LLM and Prompt Engineering.

Preferred Skills

• Strong analytical, research, and problem-solving capabilities.

Key Skills

Education

Any Gradute