Databricks Consultant

Key Responsibilities

 

• Build and enhance solutions using Databricks Workflows, Delta Lake, Unity Catalog, and Genie.
• Implement data quality checks leveraging frameworks such as DQX.
• Design and orchestrate workflows using Airflow or Databricks.
• Develop high‑quality code using PySpark, Spark SQL, Python, and SQL.
• Apply performance tuning and cost optimization best practices across pipelines.
• Leverage Genie to drive automation and productivity use cases.
• Enforce governance and security policies through Unity Catalog and monitoring practices.
• Troubleshoot and resolve issues across data pipelines to maintain high reliability.
  
   

Required Experience & Skills

 

• 5–8 years of experience in data engineering or Databricks‑focused development.
• Strong hands‑on expertise with Databricks Workflows, Delta Lake, Unity Catalog, and Genie.
• Working knowledge of data quality frameworks, ideally DQX.
• Experience with workflow orchestration using Airflow or Databricks-native tools.
• Proficiency in PySpark, Spark SQL, Python, and SQL with strong debugging and optimization skills.
• Strong problem‑solving and communication abilities with solid stakeholder engagement.
• Proven ability in pipeline performance tuning and cost optimization