Description

You will own the testing and validation of big data pipelines and cloud automation in a distributed environment.

This role is hybrid.

Responsibilities

• Design, develop, and execute automated test scripts for data pipelines and cloud applications.
• Validate data transformations and integrity across distributed systems using PySpark and SQL.
• Execute, monitor, and validate outputs for AWS EMR jobs.
• Test and troubleshoot AWS Lambda functions and related serverless workflows.
• Perform system validations using Unix shell scripts and the AWS CLI.

Required Skills

• 5+ years of experience in QA or data engineering roles.
• Proficiency in Python for scripting and automation.
• Hands-on experience with PySpark for validating distributed data processing jobs.
• Strong SQL skills for data validation and backend testing.
• Experience running and validating AWS EMR jobs.
• Experience testing AWS Lambda serverless functions.
• Competency in Unix/Linux command-line operations and scripting.
• Ability to use AWS CLI to interact with services programmatically.

Preferred Skills

• Any Graduate degree.

Key Skills

Education

Any Gradute