Description

Build and optimize Azure data platforms, managing Databricks workspaces, Delta Lake, and cloud-native ETL frameworks.

This role is remote.

Responsibilities

• Build and optimize data pipelines using Databricks, Delta Lake, and PySpark.
• Administer Databricks workspaces, including cluster policies, Unity Catalog, and Secret Scopes.
• Configure Azure Synapse, ADLS Gen2, and Key Vault, managing Linked Services and Managed Identity.
• Implement CI/CD pipelines via GitHub Actions (OIDC federation) or Azure DevOps for platform deployments.
• Automate infrastructure and platform tasks using Pulumi, Terraform, Bicep, PowerShell, and Bash.

Required Skills

• 10+ years of experience in data engineering or platform engineering.
• Strong Python and PySpark programming for data processing and automation.
• Hands-on Databricks experience: Delta Lake, Unity Catalog, Workflows, and workspace administration.
• Azure data stack proficiency: Synapse, ADLS Gen2, Key Vault, Spark Pools.
• Deep understanding of Azure fundamentals: compute, storage, networking (VNets, NSGs, Private Endpoints), and RBAC.
• Experience with CI/CD tools, specifically GitHub Actions with OIDC or Azure DevOps.
• Infrastructure-as-Code skills with Pulumi, Terraform, or Bicep across multiple environments.
• Scripting expertise in PowerShell and Bash for platform automation.
• Knowledge of monitoring tools (Azure Monitor, Log Analytics) and KQL.

Preferred Skills

• Experience with big data file formats like Parquet and Delta Lake.
• Familiarity with cloud-native data modelling and ETL/ELT frameworks on Azure.

Key Skills

Education

Any Graduate