Description

You will administer and govern the Databricks environment to support data engineering and ML workloads.

Responsibilities

• Implement data provisioning patterns based on business requirements following established policies and standards.
• Create and manage distributed workspaces, provision clusters, and manage data infrastructure sizing in Databricks.
• Develop and optimize ETL data pipelines using Databricks, focusing on data integrity and quality.
• Ensure data security and compliance by utilizing Databricks and Privacera features.
• Design and maintain data models and schemas incorporating Unity Catalog and Collibra governance.

Required Skills

• 10+ years of professional experience.
• Expertise in Databricks administration, workspace configuration, and compute resource management.
• Strong scripting proficiency in Python and SQL.
• Hands-on experience with ETL pipeline development and optimization.
• Deep knowledge of Unity Catalog administration (metastore, catalogs, schemas).
• Familiarity with security and governance, including RBAC and user management.
• Experience with data privacy tools like Privacera.
• Ability to guide data engineering teams on pipeline construction and optimization.
• Any Graduate degree.

Key Skills

Education

Any Gradute