Description

You will serve as the primary technical authority for Databricks platform projects.

Responsibilities

• Design scalable, efficient, and secure data pipelines and architectures on Databricks.
• Develop and deploy ETL/ELT pipelines and machine learning models on Databricks.
• Manage and optimize the Databricks environment, including cluster configuration and cost management.
• Ensure data security compliance and implement data governance policies within the platform.
• Review code and workflows to ensure quality, efficiency, and adherence to standards.

Required Skills

• 10+ years of professional experience in data engineering leadership roles.
• Expert proficiency in Python for data processing.
• Deep experience with Apache Spark for large-scale data workloads.
• Proven experience designing data pipelines using Databricks.
• Experience optimizing Spark jobs in Databricks for performance and cost efficiency.
• Knowledge of Data Lakehouse Architecture and Delta Lake.
• Experience with ETL Processes and data quality/governance in Databricks.
• Ability to translate business requirements into technical solutions for stakeholders.

Key Skills

Education

Any Graduate