Description

You will support data mitigation efforts on large-scale datasets (image, video, text) used by FAIR research teams to proactively mitigate data risks.

Responsibilities

• Preprocess original datasets into formats consumable by mitigation pipelines.
• Run filtering processes utilizing Integrity's pipeline.
• Consume filtering results to update original datasets, repackage, and re-ingest data.
• Identify and implement optimizations to improve data processing workflows.

Required Skills

• 5+ years of professional experience.
• Proficiency in Python for scripting file processing and data transfer automation.
• Experience building and managing data pipelines.
• Skills in data processing, cleaning, transformation, and formatting.
• Experience with SQL and processing large datasets.
• Demonstrated ability to perform data quality control and validation.
• Effective communication for collaboration with stakeholders.

Key Skills

Education

Any Graduate