Description

Strong Python and PySpark/Spark SQLdata testing experience, especially for data validation and comparison
Hands-on Databricks experience, including notebooks, Delta Lake, Unity Catalog, and job scheduling
Strong SQL skills for source-to-target data validation
Experience building automated data testing frameworks, not just manual QA
Experience with data validation such asrecord-level comparison, schema validation, count validation, data reconciliation, and column-level mismatch analysis
Experience testing data pipelines and validating source-to-target mappings
Ability to create automated test reports and investigate data discrepancies


Nice-to-Have:
Familiarity with Delta Lake and Unity Catalog
Experience with CI/CD for data pipelines
Understanding of vehicle data domains

Key Skills

Education

Bachelor's degree