Description

You ensure data integrity and model accuracy through rigorous automated testing within big data environments. You own the release quality for data models and ETL processes.

This role is on-site.

Responsibilities

• Collaborate with Data Scientists and Engineers to develop QA test suites for data models and ETL processes.
• Select and integrate data testing tools and frameworks to meet technical requirements.
• Design and implement automated tests to increase coverage and efficiency across the data pipeline.
• Monitor system performance and advise on necessary infrastructure or cluster adjustments.
• Troubleshoot application defects and manage the full defect life cycle in an Agile environment.

Required Skills

• 5+ years of experience in data testing or quality engineering.
• Advanced Python programming for creating comprehensive test suites.
• Expertise in writing complex SQL queries for data extraction and validation.
• Proficiency in Hive QL and Spark QL.
• Hands-on experience with data testing tools such as DBT, iCEDQ, QuerySurge, Denodo, or Informatica.
• Strong understanding of Big Data concepts and NoSQL databases.
• Experience with Linux or Unix environments.
• Working knowledge of messaging systems like Kafka or RabbitMQ.
• Ability to conduct User Acceptance Testing (UAT) with business users.

Preferred Skills

• Experience with BI tools including PowerBI or Tableau.
• Knowledge of Machine Learning and Statistical Analysis.

Key Skills

Education

Bachelor's degree in Computer Science