Description

You will own the validation and integrity of data pipelines within Azure and Databricks environments.

Responsibilities

• Develop and implement test strategies, test cases, and automation scripts to validate ETL/ELT pipelines.
• Perform data validation, reconciliation, and comparative analysis between source and target systems using STM.
• Monitor data quality across Delta tables and Data Warehouses to identify and resolve anomalies.
• Integrate data testing into CI/CD workflows using DevOps tools.
• Provide reporting on test execution results, defect metrics, and overall QA health.

Required Skills

• 5+ years of experience in data QA and validation on cloud-based platforms.
• Proficiency in SQL and Python scripting.
• Hands-on experience with Azure Data Factory (ADF) and Databricks.
• Experience with data profiling, reconciliation, and schema validation.
• Practical understanding of SCD Type 2 and data transformation logic.
• Experience working with large datasets, performance testing, and data lineage tools.
• Knowledge of Azure DevOps or GitHub Actions for CI/CD integration.
• Any Graduate degree.

Key Skills

Education

Any Graduate