Description

Design and develop enterprise data solutions on Azure using Databricks, Spark, Python, and SQL.

Responsibilities

  • Build and optimize Spark/PySpark pipelines, resolving performance issues like data skew and shuffles.
  • Maintain Delta Lake tables and data models for analytical and operational workloads.
  • Automate cloud infrastructure and CI/CD workflows using Terraform, GitHub, and GitHub Actions.
  • Collaborate with engineering teams to improve data security and system scalability.
  • Support Azure Functions, messaging services, and orchestration tools.

Required Skills

  • 5+ years of experience as a Data Engineer.
  • Hands-on expertise in Azure Databricks, Spark, and Python.
  • Strong SQL and database background.
  • Experience with Delta Live Tables (DLT) or Databricks SQL.
  • Proficiency with Terraform for infrastructure provisioning.
  • Experience with GitHub for version control and CI/CD automation.

Preferred Skills

  • Familiarity with data governance, lineage, or cataloging tools like Purview or Unity Catalog.

Key Skills
Education

Any Graduate

Back To Jobs