Description

You will lead the design and implementation of scalable data processing solutions using Azure Databricks, managing the full development lifecycle from planning to deployment.

You will mentor a team of technical professionals, conduct code reviews, and ensure adherence to best practices for code quality and project delivery.

Responsibilities

• Design and implement efficient data pipelines and data lake solutions using Azure Databricks and Spark.
• Lead and mentor technical staff, fostering collaboration and managing multiple projects simultaneously.
• Translate business requirements into technical specifications and facilitate communication between technical and non-technical stakeholders.
• Oversee the development lifecycle, including risk mitigation, deployment, and maintenance of data services.

Required Skills

• 7-10 years of experience in data engineering or software development.
• Proven expertise in Azure Databricks, including Spark, data pipelines, and data lakes.
• Strong programming skills in Python, Scala, or SQL.
• Experience with Azure Data Lake Storage and Azure SQL Database.
• Solid understanding of data modeling, ETL processes, and data warehousing concepts.
• Demonstrated ability to lead technical teams and deliver successful projects.

Preferred Skills

• Familiarity with Azure DevOps for CI/CD processes.
• Experience with machine learning frameworks and libraries.

Key Skills

Education

Bachelor's degree