Description

You will own the testing and validation of ETL and batch data processes.

This role is on-site.

Responsibilities

• Analyze and test data within ETL and batch processing workflows.
• Generate detailed test reports to verify data quality checks.
• Query file-based systems using Athena and Jupyter Notebooks.
• Validate data integrity across various AWS services.

Required Skills

• 10+ years of hands-on experience with Python and Spark.
• Proficiency in SQL for testing data pipelines.
• Experience utilizing AWS services including S3 and EC2.
• Hands-on experience with Athena and Jupyter Notebooks.
• Familiarity with Databricks.

Preferred Skills

• Exposure to Delta tables and Unity Catalog concepts.
• Knowledge of Machine learning model output testing.

Key Skills

Education

Any Graduate