Description

You will define and own the cross-team test architecture and frameworks for ETL/ELT, Databricks/Spark, APIs, and UIs. You will design data validation strategies for large-scale telemetry processing, including schema validation, statistical checks, and drift detection. You will architect CI/CD-integrated automated testing pipelines using Azure DevOps or GitHub Actions to support pre-deploy gating, contract testing, and regression protection.

Responsibilities

• Lead synthetic test data generation and robust mocking strategies for downstream dependencies.
• Drive performance, scalability, and distributed load testing for Spark jobs and microservices.
• Collaborate with data engineering and DevOps to define acceptance criteria and release quality gates.
• Implement observability for test executions, publishing automation health metrics and maintaining dashboards.
• Own test documentation, runbooks, and on-call handover for production incidents related to data correctness.

Required Skills

• 8+ years of professional software engineering and testing experience with technical ownership of QA practices.
• 3+ years leading or mentoring engineering/QA teams in technical areas.
• Deep knowledge of data engineering and lakehouse patterns: partitioning, ACID/Delta semantics, schema evolution, CDC, and data lineage.
• Proven experience architecting scalable automated test frameworks embedded into enterprise CI/CD using Azure DevOps or GitHub Actions.
• Strong software engineering skills with extensive coding experience in Python, Java, or Scala for test development.
• Extensive experience testing data pipelines and big-data processing (Databricks, Spark/PySpark, Delta Lake, ADLS Gen2) at scale.

Key Skills

Education

Any Gradute