Description

You will design, build, and maintain enterprise data solutions on Azure using Databricks, Spark, and Python.

This role is on-site.

Responsibilities

• Develop, optimize, and maintain Spark/PySpark data pipelines, addressing data skew and shuffle performance.
• Build and support Delta Lake tables and data models for analytical and operational use cases.
• Provision and manage cloud and Databricks resources using Terraform and Infrastructure as Code practices.
• Implement CI/CD workflows with GitHub and GitHub Actions for pipeline deployment.
• Troubleshoot failures and improve reliability across Databricks jobs, clusters, and data pipelines.

Required Skills

• 5+ years of experience as a Data Engineer.
• Hands-on experience with Azure Databricks, Spark, and Python.
• Strong SQL and database background.
• Experience with Azure cloud data services integration.
• Experience with GitHub and GitHub Actions for version control and CI/CD automation.
• Knowledge of distributed computing concepts (partitions, joins, shuffles).
• Experience with Terraform for cloud infrastructure provisioning.

Preferred Skills

• Familiarity with data governance or cataloging tools (e.g., Unity Catalog).
• Understanding of modern data engineering principles.

Key Skills

Education

Any Gradute