Description

Design and maintain scalable, secure Databricks environments on Azure. You will own the architecture, implementation, and security of data solutions, from on-prem ingestion to cloud-based lakehouse pipelines.

This role is on-site.

Responsibilities

• Design and implement Databricks clusters, medallion architecture pipelines, and API ingestion flows.
• Manage role-based access and data security using Unity Catalog.
• Oversee on-prem data ingestion into Azure using Azure Data Factory (ADF).
• Provide technical mentorship, escalated troubleshooting, and cluster health reviews.

Required Skills

• 5+ years implementing Lakehouse architecture using Databricks Data Engineering, SQL Analytics, Delta Lake, and Unity Catalog.
• 6+ years proficiency in Python, SQL, and PySpark for data integration and performance tuning.
• 7+ years experience with SQL Server features, including SSIS and CDC.
• 4+ years configuring Azure infrastructure and building large ingestion pipelines via ADF.
• 5+ years expertise in Relational and Dimensional modeling.
• 2+ years experience with Databricks platform security and data access control mechanisms.
• 2+ years implementing serverless real-time or near real-time architectures using Cloud (Azure, AWS, or GCP) and Spark Streaming/ML.
• Experience with GIT code versioning.

Preferred Skills

• Local to the Triangle, NC region and able to come onsite for equipment collection and monthly meetings.

Key Skills

Education

Bachelor's degree in Computer Science