Description

Key Responsibilities:

• Architect and manage Databricks environments, Unity Catalog, and Delta Lake
• Design and optimize data pipelines, workflows, and automation frameworks
• Monitor DBU usage, forecasting, and cost optimization strategies
• Support Unix/Linux administration, shell scripting, and platform operations
• Collaborate with Data Engineering, Analytics, Infrastructure, and Security teams
• Implement governance, monitoring, HA/DR, and compliance best practices

Required Skills:

• 7+ years in Data Engineering/Data Platforms with 4+ years of Databricks experience
• Expertise in Databricks, PySpark, Python, Delta Lake, DLT, and SQL Warehouses
• Strong Unix/Linux administration and Bash scripting experience
• Experience with AWS/Azure/GCP cloud platforms
• Knowledge of MLflow, AutoML, AI/ML concepts, and automation frameworks
• Experience with Oracle, Git, Jira, ServiceNow, and ETL tools like Fivetran/Ab Initio

Preferred:

• Experience with Generative AI/LLM workflows
• Terraform/Ansible and enterprise orchestration tools
• Insurance or financial domain experience

Key Skills

Education

Bachelor's degree