Description

You will manage and optimize data platforms within an Azure environment to support BI and analytics workflows.

Responsibilities

• Automate migrations of data engineering workflows and processes.
• Design and implement data pipelines using Azure DevOps and PowerShell.
• Manage and configure Azure Databricks workspaces, including clusters and libraries.
• Develop and deploy DataStage solutions from ideation through to production.
• Integrate data engineering workflows with Azure Cloud Engineering via CI/CD.

Required Skills

• 5+ years of experience in data engineering or platform roles.
• Proficiency in SQL or a relevant coding language.
• Hands-on experience with Azure Data Factory, including linked services, triggers, and monitoring.
• Working knowledge of PySpark and Python for data transformation.
• Experience with Azure Databricks, specifically job clusters and ETL/ELT processes.
• Demonstrable experience as a DataStage developer.
• Strong understanding of CI/CD DevOps and integrating ADF.
• Knowledge of data modeling, database design, and the data warehousing ecosystem.
• Ability to research and solve complex problems using diverse information sources.

Key Skills

Education

Any Graduate