Description

You will build and manage efficient ETL pipelines and data distribution extracts within a healthcare data environment.

Responsibilities

• Build and manage ETL pipelines using Databricks workflows or other orchestration frameworks.
• Perform data analysis and mapping to meet specific client output requirements.
• Develop data distribution extracts and associated scripts.
• Optimize the performance of data extract scripts and SQL/SparkSQL queries.
• Work with clients and implementation teams to define data distribution requirements.

Required Skills

• 10+ years of experience in data engineering.
• Proficiency with Python and FHIR standards.
• Extensive experience with Databricks and PySpark for processing structured and semi-structured data.
• Strong SQL skills, including query tuning and performance optimization in SparkSQL.
• Hands-on experience with AWS services such as S3 and SQS.
• Experience with Snowflake or similar data warehousing technologies.
• Practical knowledge of Terraform and GitLab CI/CD.
• Mandatory experience in the healthcare domain.
• Background working for a cloud-based data services provider for large healthcare clients.

Preferred Skills

• Experience managing data ingestion for both structured and semi-structured datasets.

Key Skills

Education

Any Graduate