Description

Build and manage data pipelines and transformations within the Azure ecosystem.

Responsibilities

• Implement data requirements using Azure Data Factory and Databricks.
• Perform data transformations using PySpark and Spark frameworks.
• Coordinate with business stakeholders to translate requirements into technical solutions.
• Manage version control and deployments using Git or Bitbucket within DevOps and Agile environments.
• Mentor team members and provide technical thought leadership.

Required Skills

• 5+ years of experience with Databricks.
• 3+ years of hands-on experience in PySpark for data transformations.
• Proficiency in Python or Scala.
• Strong experience with Azure Data Factory, Azure SQL, and Azure Blob Storage.
• Knowledge of Delta Lake, Azure Synapse, Azure Functions, and Azure Logic Apps.
• Ability to write complex SQL queries and apply DWH concepts.
• Experience with Hive and big data concepts.
• Familiarity with Azure Purview and Batch Account configuration.
• Experience with Git or Bitbucket for version control.

Preferred Skills

• Experience working in DevOps and Agile methodologies.

Key Skills

Education

Any Graduate