Description

You will design and maintain scalable data pipelines for healthcare clients using cloud-native technologies.

Responsibilities

• Build and manage efficient ETL pipelines using Databricks workflows and orchestration frameworks.
• Ingest and process structured and semi-structured data using Spark and SQL.
• Optimize performance of data extract scripts and queries in SQL and Spark SQL.
• Perform root cause analysis on data distribution requirements and implementation issues.
• Create data analysis and mapping specifications to produce accurate client outputs.

Required Skills

• 5+ years of experience with Python and SQL.
• Proficiency with Spark and Spark SQL for large-scale data processing.
• Experience building ETL pipelines on Databricks.
• Strong understanding of AWS services (S3, SQS) or similar cloud infrastructure.
• Practical knowledge of Snowflake or similar data warehousing solutions.
• Experience with Terraform and GitLab CI/CD for infrastructure and deployment.
• Familiarity with Agile development methodologies.

Preferred Skills

• Experience in healthcare data services with FHIR standards.
• Proven ability to communicate technical concepts to non-technical stakeholders.

Key Skills

Education

Bachelor's degree