Description

Lead the implementation and maintenance of Azure Databricks environments to support data processing requirements.

Responsibilities

• Implement and maintain Azure Databricks infrastructure for scalable data processing.
• Process, clean, and transform raw data into usable formats for stakeholders and data scientists.
• Optimize system efficiency and effectiveness through continuous technical improvements.
• Collaborate with data scientists to ensure data accessibility and utility across the platform.

Required Skills

• 9+ years of experience in data engineering or related roles.
• Deep expertise in Azure Databricks and the Microsoft cloud platform.
• Proficiency in SQL for managing relational database management systems.
• Strong hands-on experience with Python or Scala.
• Solid understanding of Apache Spark for large-scale data processing.
• Practical knowledge of ETL processes and Hadoop frameworks.
• Experience with cloud migration and Bi360.
• Bachelor's Degree in Computer Science, IT, Data Science, or a related field.

Preferred Skills

• Familiarity with machine learning algorithms and concepts.

Key Skills

Education

Bachelor's Degree