Description

You will design, build, and maintain end-to-end data pipelines on Google Cloud Platform. You optimize large-scale data processing using DBT, SQL, and Python. You manage containerized deployments and secure credential management.

Responsibilities

• Build and optimize data transformation processes using DBT, SQL, and Python.
• Leverage GCP services including BigQuery, Dataflow, Cloud Composer, and Cloud Functions for scalable solutions.
• Implement CI/CD pipelines using Codefresh and GitHub Actions alongside infrastructure automation with Terraform.
• Monitor and troubleshoot data pipelines to ensure system reliability and efficiency.
• Collaborate with data scientists and engineers to support data-driven decision-making and governance.

Required Skills

• 10+ years of experience in data engineering roles.
• Extensive experience with GCP services: BigQuery, Dataflow, Cloud Composer, Cloud Functions.
• Strong expertise in DBT for data transformation and modelling.
• Proficiency in Python, Java, and SQL.
• Infrastructure as Code experience with Terraform.
• Experience with CI/CD tools: Codefresh, GitHub Actions.
• Knowledge of containerization (Docker) and HashiCorp Vault.

Preferred Skills

• Experience with AI, Large Language Models (LLMs), and Machine Learning.
• Familiarity with ML frameworks such as TensorFlow, PyTorch, or Scikit-learn.

Key Skills

Education

Any Graduate