Description

You will manage and automate the deployment lifecycles for data-centric cloud infrastructure and workflows.

Responsibilities

• Design and maintain CI/CD pipelines for Azure Data Factory and Databricks notebooks.
• Automate unit, integration, and QA testing workflows within Databricks.
• Develop Azure DevOps Pipelines using YAML builds and release workflows.
• Write automation and infrastructure management scripts using Python, PowerShell, and Terraform.
• Implement CI/CD pipelines for schema updates and manage cloud resources via infrastructure-as-code.
• Monitor and troubleshoot pipeline failures and deployment inconsistencies to ensure data pipeline reliability.

Required Skills

• 12+ years of professional experience.
• Azure DevOps (ADO) and YAML-based pipeline development.
• Azure Data Factory (ADF) and Databricks.
• Python and PowerShell scripting.
• Terraform for infrastructure management.
• Experience with DABs, ADLS Gen2, and Key Vault.
• Strong understanding of data architecture and schema update workflows.
• Ability to collaborate with Data Engineers and Platform teams.

Key Skills

Education

Any Gradute