Description

You will build and maintain data processing pipelines within the Azure ecosystem.

Responsibilities

• Develop Databricks notebooks to join, filter, pre-aggregate, and process files stored in Azure Data Lake.
• Perform data enrichment on raw data and create parameterized notebooks for processing facts and dimensions.
• Implement data transformation and processing logic using PySpark.
• Design and maintain systems using Azure Cloud, ADF, Azure SQL DB, and Azure Synapse.

Required Skills

• 5+ years of experience in data engineering or database management.
• Proficiency in Python, SQL, PySpark, and Scala.
• Experience with Azure Cloud services including Azure Data Lake Gen2, Azure Databricks, and Azure DevOps.
• Hands-on use of Azure Data Factory (ADF), Azure Synapse, and Azure Logic Apps.
• Competency with Azure SQL DB, Azure SQL Server, and SSMS.
• Experience using Git, Visual Studio, and Visual Studio Code.
• Ability to work with Jupyter Notebooks and Power BI.
• Knowledge of Power Apps, Power Automate, and Power Flow.

Preferred Skills

• Experience building applications based on specific user requirements using the Azure stack.

Key Skills

Education

Any Graduate