Description

You will define and implement enterprise-level testing strategies and data quality frameworks for large-scale data platforms.

This role is remote.

Responsibilities

• Design testing approaches for ETL/ELT pipelines, batch processing, and streaming systems.
• Establish data quality, validation, and governance frameworks including reconciliation and lineage validation.
• Enable and scale test automation across big data ecosystems like Databricks and Snowflake.
• Implement data observability and profiling practices to proactively monitor data health.
• Lead defect analysis, continuous improvement of pipeline reliability, and mentor engineering teams.

Required Skills

• 10–15+ years of experience in data engineering, QA, or data validation/testing.
• Strong hands-on expertise in SQL and Python.
• Proven experience with data testing frameworks, specifically Great Expectations.
• Experience with large-scale data platforms and distributed systems (Databricks, Snowflake, Spark, Hadoop).
• Solid understanding of data modeling concepts (dimensional modeling, Lakehouse patterns) and governance.
• Experience with cloud platforms such as Azure or AWS and CI/CD automation practices.
• Excellent client communication skills with the ability to articulate complex data quality concepts.
• Bachelor's degree in a relevant field.

Preferred Skills

• Experience with data mesh or Lakehouse architectures and real-time processing frameworks (Kafka, Event Hubs).
• Understanding of DevOps/DataOps practices and performance testing of data systems.

Key Skills

Education

Bachelor's degree