Description

Lead the design and implementation of Databricks Lakehouse architectures for healthcare data.

Responsibilities

• Design Databricks Lakehouse architecture using Bronze, Silver, and Gold layers.
• Define ingestion patterns for API and FTP sources.
• Translate legacy SQL Server logic into scalable Spark-based transformations.
• Establish Delta Lake optimization strategies, security protocols, and PHI governance controls.
• Implement CI/CD pipelines for data operations and mentor the engineering team.

Required Skills

• Strong expertise in Databricks and Delta Lake.
• Advanced proficiency in PySpark and Spark SQL.
• Experience designing data pipelines from scratch.
• Deep understanding of SQL Server logic and stored procedures.
• Hands-on experience with Azure cloud services.
• 5+ years of experience in data engineering or architecture.

Preferred Skills

• Experience with healthcare data architecture and PHI compliance.

Key Skills

Education

Any Gradute