Description
You will own data quality and testing for ETL pipelines, Databricks/Spark platforms, and AI/ML models.
This role is on-site.
Responsibilities
- Execute end-to-end ETL and data testing to ensure pipeline accuracy and integrity.
- Validate Databricks and Spark-based data platforms for performance and correctness.
- Perform AI/ML model testing and MLOps validation to verify model outputs and deployment stability.
- Develop and maintain test automation scripts using Python and SQL.
- Conduct integration testing across data sources and downstream consumers.
Required Skills
- 6+ years of total experience in ETL and data testing.
- 3+ years of hands-on experience testing Databricks or Spark-based platforms.
- Strong coding proficiency in Python and SQL for test development.
- Proven experience in AI/ML model testing or MLOps validation.
- Experience with test automation frameworks and tools.
- Solid understanding of integration testing principles and data workflows.
Preferred Skills
- Experience with large-scale data warehousing and cloud data platforms.
- Familiarity with CI/CD pipelines for data and ML models.