Description

You will design and implement DevOps and DataOps processes for Azure-based cloud data warehouses and data pipeline platforms.

Responsibilities

• Establish DevOps standards for data platforms, including code repository folder structures and branching strategies.
• Build and optimize data pipelines using ETL and ELT patterns to extract data from multiple sources into data warehouses.
• Automate manual processes such as data extraction, testing, and deployment to increase reliability and reduce development time.
• Design and execute automated testing at every pipeline stage, including unit, performance, and end-to-end tests.
• Collaborate with infrastructure engineers on IaC design for KMS, secrets, roles, policies, clusters, and pipelines.

Required Skills

• 12+ years of professional experience.
• Proven experience as a DevOps Architect with at least two major Azure DevOps implementations.
• Direct experience implementing DevOps within data engineering projects.
• Hands-on expertise with the Azure stack, specifically Azure Data Factory (ADF) and Databricks.
• Experience using ARM templates and Infrastructure as Code (IaC) automation.
• Deep understanding of Azure security and networking configurations.
• Proficiency in SQL and managing SQL artifacts.
• Practical knowledge of ELT and ETL processes.
• Experience managing production data pipelines.

Key Skills

Education

Any Graduate