Description

Key Responsibilities

• Own and drive the overall enterprise data architecture strategy and roadmap.
• Design and implement scalable, secure, and high-performance data platforms on AWS and Databricks.
• Lead end-to-end data modernization and cloud transformation initiatives.
• Architect and oversee Lakehouse implementations using Medallion Architecture (Bronze, Silver, and Gold layers).
• Design metadata-driven, reusable, and scalable data ingestion and transformation frameworks.
• Collaborate with business stakeholders, product owners, architects, and executive leadership to define data solutions aligned with business objectives.
• Lead architecture reviews, establish best practices, and ensure adherence to enterprise standards.
• Design and implement real-time, batch, streaming, and CDC-based data integration solutions.
• Establish enterprise data governance, lineage, observability, security, and compliance frameworks.
• Provide technical leadership to onsite and offshore data engineering teams.
• Drive performance tuning, cost optimization, scalability, and reliability initiatives across data platforms.
• Ensure healthcare and regulatory compliance requirements are incorporated into solution designs.
• Support production operations, troubleshooting, and continuous improvement activities.

Required Skills & Experience

1. 12–15+ years of overall experience in Data Engineering, Data Architecture, Data Warehousing, Analytics, or Business Intelligence.
2. Strong hands-on experience with AWS cloud services and enterprise-scale Databricks implementations.
3. Proven experience serving as a Lead Architect or Principal Architect for enterprise data platform initiatives.
4. Advanced expertise in Python, PySpark, Spark SQL, Apache Spark, Delta Lake, and distributed data processing.
5. Deep hands-on experience with Databricks Architecture, Unity Catalog, Delta Live Tables (DLT), Databricks Workflows, Databricks Asset Bundles, and cluster optimization.
6. Proven experience designing and implementing enterprise-scale data pipelines and metadata-driven frameworks.
7. Extensive expertise in ETL/ELT architecture, orchestration, optimization, and data platform operations.
8. Strong knowledge of Lakehouse Architecture and Medallion Framework (Bronze, Silver, Gold).
9. Experience designing dimensional models, enterprise data warehouses, semantic layers, and analytical solutions.
10. Experience with batch processing, streaming, incremental loads, CDC, and real-time data processing frameworks.
11. Strong understanding of data quality, governance, lineage, reconciliation, metadata management, and platform observability.
12. Hands-on experience with CI/CD pipelines, source control, Infrastructure as Code, and automated deployments.
13. Strong experience architecting large-scale healthcare data platforms involving:
    • Claims Data
    • Clinical Data
    • EMR/EHR Systems
    • Patient Data
    • Provider Data
    • Member Data
14. Knowledge of healthcare interoperability standards and regulations including:
    • HL7
    • FHIR
    • ICD-10
    • CPT
    • HIPAA
15. Experience implementing enterprise data governance, MDM, data cataloging, and security frameworks.
16. Strong understanding of data security, privacy, encryption, role-based access controls, and PHI protection.
17. Experience migrating legacy data warehouses and ETL platforms to AWS and Databricks Lakehouse environments.
18. Excellent communication, stakeholder management, presentation, and customer-facing consulting skills.
19. Experience leading and mentoring cross-functional Data Engineering teams.
20. Ability to own architecture decisions from project initiation through deployment and production support.
21. Willingness and ability to work onsite and engage directly with business stakeholders.

Technical Skills

Cloud & Data Platforms

• AWS
• Databricks
• Lakehouse Architecture
• Delta Lake
• Unity Catalog
• Delta Live Tables (DLT)

Programming & Data Engineering

• Python
• PySpark
• Spark SQL
• Apache Spark
• SQL

AWS Services

• S3
• Glue
• Athena
• Redshift
• Lambda
• EMR
• IAM
• CloudWatch
• Step Functions

Data Integration

• ETL / ELT
• CDC
• Streaming
• Kafka
• Event-Driven Architectures

Governance & Security

• Data Governance
• Data Lineage
• Data Quality
• Metadata Management
• MDM
• HIPAA Compliance

DevOps

• Git
• Azure DevOps
• Jenkins
• Terraform
• CI/CD Pipelines

Reporting & Analytics

• Power BI
• Tableau
• Amazon QuickSight

Preferred Qualifications

1. Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
2. AWS Certified Solutions Architect, AWS Certified Data Engineer, or equivalent certification.
3. Databricks Certified Data Engineer Professional or Databricks Solution Architect certification.
4. Experience in Healthcare, Life Sciences, Payer, Provider, or Clinical Analytics environments.
5. Prior experience as a Lead Architect driving enterprise-wide data modernization programs.
6. Experience implementing Databricks Lakehouse solutions for Healthcare Analytics, Population Health, Clinical Reporting, or Regulatory Reporting.
7. Knowledge of HIPAA, HITECH, CMS regulations, PHI protection, and healthcare compliance standards.
8. Strong consulting and customer-facing architecture experience.
9. Experience with Agile, Scrum, or SAFe delivery methodologies.
10. Experience managing global onsite/offshore delivery teams

Key Skills

Education

Bachelor's or Master's degrees