Description

Key Responsibilities

* Define and implement an enterprise-level testing strategy for data platforms

* Establish and own data quality, validation, and governance frameworks

* Design testing approaches for ETL/ELT pipelines, batch processing, and streaming systems

* Enable and scale test automation across big data ecosystems (e.g. Databricks, Snowflake)

* Define standards for data reconciliation, lineage validation, and schema testing

* Implement and drive data observability and data profiling practices to proactively monitor data health

* Perform current state assessments of data testing and quality frameworks and define transformation roadmaps

* Propose and implement modern data testing solutions and best practices aligned with industry standards

* Partner with data/QA engineers, analysts, and business stakeholders to ensure data accuracy and usability

* Drive adoption of testing best practices and shift-left approaches within data engineering teams

* Ensure adherence to regulatory and compliance requirements and maintain data integrity

* Lead defect analysis and continuous improvement of data quality and pipeline reliability

* Build solution offerings, accelerators, and reusable frameworks for data testing and quality engineering

* Create and deliver client presentations, proposals, and solution narratives to support business development and win engagements

3. Required Skills & Qualifications

* 10–15+ years of experience in data engineering, QA, or data validation/testing

* Strong hands-on expertise in SQL and Python

* Experience working with large-scale data platforms and distributed systems

* Proven experience with data testing frameworks (e.g., Great Expectations )

* Solid understanding of data modeling concepts (dimensional modeling, normalization, Lakehouse patterns)

* Knowledge of data governance, metadata, lineage, and data quality principles

* Experience with data observability and profiling tools and techniques

* Experience with cloud platforms such as Azure or AWS

* Familiarity with CI/CD and automation practices in data pipelines

* Basic familiarity with AI-assisted development tools or intelligent data quality techniques (e.g., anomaly detection, pattern-based validations)

* Excellent client communication and presentation skills, with the ability to articulate complex data quality and testing concepts to both technical and non-technical stakeholders

* Demonstrated ability to perform current state assessments and recommend scalable, modern solutions

* Strong problem-solving and stakeholder management skills

4. Preferred Skills

* Experience with data mesh or Lakehouse architectures

* Exposure to real-time data processing frameworks (e.g., Kafka, Event Hubs)

* Understanding of DevOps/DataOps practices

* Experience in performance and scalability testing of data systems

5. Tools & Technologies

* Data Platforms: Databricks, Snowflake

* Processing Frameworks: Apache Spark, Hadoop

* Testing Frameworks: Great Expectations

* Programming & Querying: Python, SQL

* Orchestration: Airflow, Azure Data Factory, Prefect

* Cloud: Azure, AWS, GCP

* CI/CD: Jenkins, GitHub Actions, Azure DevOps

* Streaming: Kafka, Azure Event Hubs

* Governance & Lineage: Collibra, Alation, Apache Atlas

6. Leadership Expectations

* Provide architect-level ownership of data testing practices across the organization

* Define and enforce standards, frameworks, and best practices

* Mentor and guide engineering and QA teams

* Drive alignment across teams and influence data quality and reliability initiatives

* Contribute to capability building, solution development, and pre-sales support

* Promote a culture of accountability and quality in data delivery

7. KPIs / Success Metrics

* Improvement in data quality and accuracy

* Increased reliability and stability of data pipelines

* Reduction in data defects and production issues

* Growth in test automation coverage

* Adoption of data observability and profiling practices

* Faster and more predictable data validation cycles

* Contribution to solution offerings and successful client engagements

* Compliance and audit readiness

Key Skills

Education

Bachelor's degree