Description

You will design and optimize data pipelines to process large-scale healthcare datasets.

This role is on-site.

Responsibilities

• Design, implement, and optimize data pipelines using Spark with Scala and Airflow.
• Build and maintain ETL processes to transform healthcare data into common data models.
• Manage big data using GCP services including Google Cloud Storage, BigQuery, and Dataproc.
• Ensure data quality, integrity, and security with a focus on HIPAA compliance.
• Monitor and troubleshoot pipelines to ensure performance and minimal downtime.

Required Skills

• 5+ years of experience in a data engineering role.
• Proficiency with Spark and Scala.
• Experience with Airflow for workflow orchestration.
• Hands-on experience with GCP (Google Cloud Platform), GCS, and BigQuery.
• Strong understanding of ETL processes and Big Data technologies.
• Knowledge of HIPAA compliance and data security requirements.
• Ability to translate data requirements into efficient processing workflows.
• Any Graduate degree.

Key Skills

Education

Any Graduate