Description

Design and lead data engineering solutions using Databricks and Google Cloud Platform, owning the architecture, implementation, and optimization of large-scale data pipelines.

This role is on-site.

Responsibilities

• Architect and build data models, schemas, and governance frameworks.
• Develop and optimize data pipelines using Databricks (Autoloader, DLT, Delta Lake, CDF) and PySpark.
• Implement CI/CD pipelines and infrastructure-as-code using Terraform.
• Provide architectural guidance and lead technical teams.
• Evaluate and integrate Gen AI concepts into practical data applications.

Required Skills

• 10+ years of experience as a Data Engineer or Architect.
• Extensive hands-on experience with Databricks, including Autoloader, DLT, Delta Lake, and CDF.
• Strong expertise in PySpark and Python for data engineering and automation.
• Advanced SQL skills with a focus on query optimization.
• Deep proficiency with GCP services: BigQuery, Cloud Functions, Cloud Run, Pub/Sub, Dataflow, and GCS.
• Proven experience with CI/CD pipelines and DevOps practices.
• Experience with infrastructure-as-code, specifically Terraform.
• Strong leadership and stakeholder management skills.

Preferred Skills

• Experience integrating Gen AI concepts into data applications.

Key Skills

Education

Not specified