Description

Own data integrity across complex healthcare data pipelines through automation and validation.

This role is on-site.

Responsibilities

• Build automated data quality checks in PySpark for healthcare datasets.
• Develop reconciliation frameworks between legacy SQL outputs and new Databricks outputs.
• Implement rule-based validation to ensure accuracy in healthcare datasets.
• Automate profiling and anomaly detection processes within data pipelines.
• Integrate quality checks into pipeline workflows for continuous data reliability.

Required Skills

• 6+ years of experience in data quality automation and validation.
• Proficiency in SQL.
• Expertise in PySpark.
• Experience with Databricks.
• Strong background in Data Quality Automation.
• Solid understanding of Data Validation principles.
• Experience with pipeline workflows.
• Familiarity with Healthcare Data standards.

Preferred Skills

• Experience with DQ scorecards and dashboards.
• Knowledge of rule-based validation frameworks.

Key Skills

Education

Any Graduate