Description

You define and own the cross-team test architecture and frameworks for ETL/ELT, Databricks/Spark, APIs, and UIs.

Responsibilities

• Design data validation strategies for large-scale telemetry processing, including schema validation, statistical checks, and drift detection.
• Architect CI/CD-integrated automated testing pipelines using Azure DevOps or GitHub Actions to support pre-deploy gating, contract testing, and regression protection.
• Lead synthetic test data generation and robust mocking strategies for downstream dependencies.
• Drive performance, scalability, and distributed load testing for Spark jobs and microservices.
• Implement observability for test executions, publishing automation health metrics and maintaining dashboards.

Required Skills

• 8+ years of professional software engineering and testing experience with technical ownership of QA practices.
• 3+ years leading or mentoring engineering/QA teams in technical areas.
• Deep knowledge of data engineering and lakehouse patterns: partitioning, ACID/Delta semantics, schema evolution, CDC, and data lineage.
• Proven experience architecting scalable automated test frameworks embedded into enterprise CI/CD using Azure DevOps or GitHub Actions.
• Strong software engineering skills with extensive coding experience in Python, Java, or Scala for test development.
• Extensive experience testing data pipelines and big-data processing (Databricks, Spark/PySpark, Delta Lake, ADLS Gen2) at scale.
• Collaborate with data engineering and DevOps to define acceptance criteria and release quality gates.
• Own test documentation, runbooks, and on-call handover for production incidents related to data correctness.

Key Skills

Education

Any Gradute