Description

Key Skills and Experience
·      Databricks Ecosystem: Hands-on experience with Databricks Workspace, Unity Catalog, Delta Lake, Spark UI, and MLflow.
·      Programming & Data Engineering: Strong development experience with Python, PySpark, SQL, and/or Java, including the development and optimization of data processing and ETL/ELT pipelines.
·      Cloud Platforms: Strong Microsoft Azure experience, particularly Azure Databricks, Azure Data Lake Storage Gen2 (ADLS Gen2), and Azure Data Factory. Experience with AWS, including S3 and IAM, is also desirable.
·      Apache Spark: Experience developing, troubleshooting, and performance-tuning Spark-based workloads and data processing pipelines.
·      DevOps & MLOps: Experience with Git, CI/CD pipelines, OpenShift, Kubernetes, and Terraform / Infrastructure as Code (IaC).
·      Security & Governance: Experience implementing or supporting Databricks security, access controls, secrets management, identity integration, data governance, and Unity Catalog.
·      Automation & Operations: Experience with Databricks workflows, job orchestration, monitoring, troubleshooting, performance optimization, and production support.
·      Enterprise Delivery: Ability to collaborate across technical teams and operate within established enterprise architecture, security, change-management, and governance standards.
·      Knowledge of Apache Airflow
 
Preferred Experience
·      Experience designing and delivering production-grade Databricks solutions in a large enterprise environment, particularly where security, data governance, resiliency, automation, and operational support are key requirements

Key Skills

Education

Bachelor's or Master's degrees