Description

You will design and implement scalable data applications and end-to-end pipelines using Databricks.

Responsibilities

• Build and manage complex data pipelines using Spark and Hive frameworks.
• Implement Databricks Lakehouse concepts and Medallion architecture in enterprise environments.
• Execute engineering practices including code refactoring, design patterns, and CI/CD.
• Develop highly scalable data processes involving structured and unstructured data.
• Manage data warehousing tasks including dimensional modeling, ETL, and data governance.

Required Skills

• 10+ years of experience in Data Engineering.
• Proven experience implementing at least two end-to-end projects in Databricks.
• Hands-on expertise with Delta Lake, dbConnect, db API 2.0, and Databricks workflows orchestration.
• Strong coding proficiency in either Python or Scala.
• Two years of experience with Hadoop/Spark in development, administration, or support roles.
• Solid experience with SQL-based databases.
• Deep understanding of Spark and Hive data processing.
• Bachelor's degree.

Preferred Skills

• Knowledge of Unity Catalog and basic data governance.
• Experience with MicroStrategy, Tableau, Docker, Kubernetes, or Linux/Unix administration.

Key Skills

Education

Bachelor's degree