Description

You will design, optimize, and maintain cloud-based data platforms on Databricks to support advanced analytics, machine learning, and AI initiatives for customers.

This role is remote.

Responsibilities

• Consult directly with customers to design, optimize, and fine-tune Databricks environments for cost, scalability, and performance.
• Build and maintain scalable ETL/ELT pipelines using Spark and Delta Lake.
• Integrate Databricks solutions with AWS, Azure, and GCP cloud environments.
• Implement data governance, security, compliance, and data management best practices.
• Troubleshoot platform issues and drive continuous improvements for data quality and performance.

Required Skills

• 10+ years of experience with Databricks in production environments.
• Strong Python and/or SQL skills.
• Deep knowledge of Apache Spark and building/maintaining ETL/ELT pipelines.
• Experience with Data Lakehouse architecture, data modeling, and performance tuning.
• Proficiency with AWS S3, Azure Data Lake, or GCP Storage.
• Experience with CI/CD and Git workflows.
• Active Secret Clearance and U.S. Citizenship required.

Preferred Skills

• Master's Degree or PhD in a relevant field.
• Experience with MLflow, Delta Lake, and Unity Catalog.
• Familiarity with FedRAMP, HIPAA, SOC2, or similar compliance frameworks.

Key Skills

Education

Any Graduate