Description

Candidates must have production Databricks experience and should be able to confidently pass a Databricks-led technical interview.

Must Have:

• Active Secret Clearance
• Proven Databricks production environment experience
• Strong Python and/or SQL
• Strong Apache Spark experience
• ETL/ELT pipeline development experience
• Experience with AWS, Azure, or GCP data platforms
• Data Lakehouse architecture experience
• Data modeling, data quality, and performance tuning experience
• CI/CD pipelines and Git
• Willingness to travel to customer sites as needed

Nice to Have:

• MLflow
• Delta Lake
• Unity Catalog
• Kafka, Kinesis, or Event Hubs
• Tableau, Power BI, or Looker
• Terraform, ARM, or CloudFormation
• FedRAMP, HIPAA, SOC2, or other compliance experience
• DevOps/MLOps experience
• Master's Degree or PhD
• Prior consulting or customer-facing experience

What We Are Really Looking For:

• Candidates actively building and optimizing Databricks environments, not simply users of the platform
• Heavy data engineering backgrounds preferred
• Individuals who have optimized Databricks clusters, workflows, architecture, and performance
• Strong customer-facing consultants who can advise clients and design solutions
• Candidates who can speak fluently about Spark, ETL/ELT, Lakehouse architecture, governance, and cloud integrations

Interview Process:

• Interview with Mike Iarrobino
• Technical interview with Databricks
• Final hiring decision made by Databricks

Formal Job Description

Seeking a Sr. Databricks Consultant to work remotely supporting Professional Services and Managed Cybersecurity Services programs. This position is contingent upon additional funding.

The ideal candidate will possess deep expertise within the Databricks ecosystem and will consult directly with customers to design, optimize, and fine-tune cloud-based data platforms supporting advanced analytics, machine learning, and AI initiatives.

Key Responsibilities

• Consult directly with customers to design, optimize, and fine-tune Databricks environments
• Design, build, and maintain scalable ETL/ELT pipelines
• Optimize Databricks clusters, workflows, and architecture for cost, scalability, and performance
• Integrate Databricks with AWS, Azure, and GCP environments
• Implement data governance, security, compliance, and data management best practices
• Utilize Spark, Delta Lake, MLflow, and Databricks Machine Learning technologies
• Collaborate with engineers, analysts, data scientists, and business stakeholders
• Troubleshoot platform and infrastructure issues while driving continuous improvements
• Support advanced analytics, machine learning, and AI-focused customer initiatives

Required Qualifications

• Active Secret Clearance
• Proven experience with Databricks in production environments
• Strong Python and/or SQL skills
• Apache Spark experience
• Experience building and maintaining ETL/ELT pipelines
• Knowledge of Data Lakehouse architecture
• Experience with AWS S3, Azure Data Lake, or GCP Storage
• Data modeling, data quality, and performance tuning experience
• CI/CD and Git experience
• Bachelor's degree preferred (experience considered in lieu of degree)
• Ability to travel as needed

Preferred Qualifications

• Master's Degree or PhD
• MLflow experience
• Delta Lake experience
• Unity Catalog experience
• Kafka, Kinesis, or Event Hubs
• Tableau, Power BI, or Looker
• Terraform, ARM, or CloudFormation
• FedRAMP, HIPAA, SOC2, or similar compliance experience
• DevOps/MLOps experience
• Prior consulting or customer-facing experience
• Agile project delivery experience and familiarity with professional services environments

Key Skills

Education

Any Graduate