Description

You will lead testing efforts for data platforms, focusing on data quality, pipeline integrity, and automation.

Responsibilities

• Analyze technical requirements to ensure test coverage across data mapping, pipelines, transformations, and reporting.
• Design and execute test plans for Databricks notebooks, ETL workflows, and orchestration jobs.
• Validate data ingestion, transformation, aggregation, and cleansing logic against defined business rules.
• Develop and maintain automated testing frameworks using SQL and Python.
• Integrate automated tests into CI/CD pipelines and report execution results to stakeholders.

Required Skills

• 8+ years of relevant testing experience.
• Deep experience with Databricks testing (notebooks, ETL workflows, and Orchestration jobs).
• Proficiency in Python and SQL for data validation and scripting.
• Experience with testing frameworks such as Pytest, JUnit, or TestNG.
• Ability to integrate automated tests into CI/CD pipelines (Azure DevOps, Jenkins, or GitHub Actions).
• Experience testing data quality dimensions: accuracy, completeness, consistency, timeliness, and validity.
• Familiarity with data governance and security policies (e.g., Unity Catalog).

Key Skills

Education

Bachelor's degree