Description

Lead the design, implementation, and optimization of the Databricks platform. Own data pipelines, ETL processes, and data models. Ensure platform performance, scalability, and reliability. Implement security measures for data privacy and compliance. Troubleshoot performance issues, system errors, and data inconsistencies. Manage the team and provide guidance to junior members.

Responsibilities

• Design and implement scalable data pipelines and ETL processes using Databricks and Apache Spark.
• Develop and maintain data models, warehousing structures, and integration techniques.
• Implement security measures to ensure data privacy, governance, and quality compliance.
• Troubleshoot and resolve performance bottlenecks, system errors, and data inconsistencies.
• Manage the team, providing technical guidance and mentorship to junior engineers.

Required Skills

• 5+ years of experience in data engineering or architecture roles.
• Hands-on experience with the Databricks platform and Apache Spark.
• Proficiency in Python, Scala, or SQL.
• Familiarity with cloud platforms such as AWS or Azure.
• Strong understanding of data engineering principles, data modeling, and data warehousing.
• Bachelor's degree in Computer Science, Engineering, or a related field.

Preferred Skills

• Experience with Snowflake data platform.
• Master's degree in Computer Science, Engineering, or a related field.
• Knowledge of data governance and data quality frameworks.

Key Skills

Education

Any Graduate