Description

You will own the quality assurance and automation strategy for data processing pipelines within a Big Data ecosystem.

Responsibilities

• Develop and implement comprehensive test strategies for complex data initiatives.
• Build, design, and execute automation frameworks and scripts using Python or PySpark.
• Estimate, prioritize, and coordinate testing activities within an Agile team.
• Perform root cause analysis, troubleshoot data issues, and conduct thorough regression testing.
• Track quality metrics such as defect densities and open defect counts while providing oversight to junior team members.

Required Skills

• 8+ years of IT experience with at least 5 years focused on ETL testing and automation.
• Proficiency in Python and PySpark for developing automation frameworks.
• Advanced SQL skills for handling complex queries and big data validation.
• Hands-on experience with Informatica and ETL toolsets.
• Experience testing in AWS cloud environments, specifically using S3 and Redshift.
• Practical knowledge of DevOps practices applied to ETL and Cloud workflows.
• Experience working within a data lakehouse architecture.
• Strong ability to review technical design documents and provide feedback on requirements.
• Effective communication skills for interacting with business and cross-functional teams.

Preferred Skills

• Familiarity with DBT (Data Build Tool) or similar data transformation tools.

Key Skills

Education

Any Graduate