Description

You will build and maintain data pipelines using Azure Databricks and PySpark.

This role is on-site.

Responsibilities

• Implement data transformations using PySpark and Python within Azure Databricks.
• Design and manage data workflows using Azure Data Factory and Logic Apps.
• Coordinate with business stakeholders to translate requirements into technical implementations.
• Manage version control and deployment processes using Git or Bitbucket within Agile environments.
• Configure storage and monitoring options for Azure Blob storage and Data Lake accounts.

Required Skills

• 8+ years of overall experience in data engineering.
• 5+ years of hands-on experience with PySpark for data transformations.
• Proficiency in Python or Scala.
• Extensive experience with Azure Databricks, Azure Data Lake, and Delta Lake.
• Strong knowledge of Azure Data Factory, Azure Synapse, and Azure Functions.
• Ability to write complex SQL queries and apply DWH concepts.
• Experience with Azure Purview, Azure SQL, and Azure Logic Apps.
• Understanding of big data frameworks including Hive and Spark.
• Practical knowledge of Git or Bitbucket for version control.

Key Skills

Education

Any Graduate