Description

About Position
We are seeking a highly skilled Senior Data Engineer to lead our cloud data engineering initiatives, focusing on the migration from SQL Server to Databricks and Snowflake. This role requires a deep understanding of data infrastructure, cloud technologies, and best practices in DevOps. You will collaborate with cross-functional teams to ensure that our data architecture supports the needs of our organization while maintaining compliance with industry standards.

What You'll Do
Lead the migration of SQL Server to Databricks and Snowflake, including schema conversion, ETL re-platforming, data validation, and cutover planning.
Design and maintain Databricks infrastructure, including Unity Catalog, Bronze/Silver/Gold medallion architecture, cluster/job orchestration, and cost optimization.
Build and manage Snowflake environments, focusing on warehouses, RBAC, data sharing, and performance tuning.
Own CI/CD pipelines for data engineering workflows using Azure DevOps, GitHub Actions, or similar tools, ensuring automated testing, deployment, and rollback.
Implement Infrastructure as Code (Terraform/Bicep) for repeatable and auditable environment provisioning.
Establish DevOps best practices, including branching strategy, code review gates, and environment promotion (dev/test/prod).
Build monitoring, alerting, and observability for pipeline health and data quality.
Collaborate with data engineers, analysts, and data scientists across various scrum teams to ensure infrastructure supports downstream needs.
Ensure HIPAA-compliant security posture across cloud environments, including encryption, access controls, and audit logging.
Mandatory Skills
5+ years of experience in cloud data engineering/DevOps, with hands-on experience in Databricks and Snowflake in production.
Strong CI/CD pipeline design experience using Azure DevOps, GitHub Actions, Jenkins, or GitLab CI.
Proven experience in SQL Server-to-cloud migration.
Proficiency in Infrastructure as Code (Terraform, Bicep, or CloudFormation).
Fluency in Python and SQL; PySpark experience is strongly preferred.
Solid understanding of medallion architecture, data lineage, and governance patterns.
Experience with Azure (preferred) or AWS/GCP cloud platforms.
Preferred Skills
Healthcare/payer data experience (HEDIS, claims, clinical data).
Exposure to Databricks Unity Catalog and MLflow.
Familiarity with Genie/AI-BI or similar natural-language query tools.
Expertise You'll Bring
As a Senior Data Engineer, you will bring a wealth of experience in cloud data engineering and a passion for building robust data infrastructures. Your ability to collaborate with diverse teams and your commitment to best practices will be essential in driving our data initiatives forward.

Benefits:

Competitive salary and benefits package
Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
Opportunity to work with cutting-edge technologies
Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
Annual health check-ups
Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents
 

Key Skills

Education

Any Graduate