Description

You will design, develop, and execute test strategies for the Big Data platform, ensuring quality, reliability, and performance across distributed systems.

This role is on-site.

Responsibilities

• Develop and maintain automated test scripts using Python and pytest/unittest for comprehensive coverage.
• Utilize PySpark and Spark SQL to validate data transformations and integrity within Big Data pipelines.
• Design and execute test plans for functional, integration, regression, and performance testing.
• Collaborate with developers and data scientists to analyze, debug, and resolve software defects.
• Analyze large datasets to identify data quality issues and validate ETL processes.

Required Skills

• 5+ years of experience in QA testing, with strong hands-on Python scripting skills.
• Proficiency in PySpark and Spark SQL for distributed system validation.
• Experience with cloud testing on AWS, specifically S3, EMR, Glue, and Lambda.
• Knowledge of CI/CD pipelines and test automation integration.
• Understanding of data validation techniques and ETL testing.
• Familiarity with RDBMS SQL and traditional SQL concepts.
• Bachelor's degree in Computer Science or a related field.

Key Skills

Education

Any Graduate