Description

You will develop and implement data pipelines to extract, transform, and load ETL data from various sources into Azure data services.

Responsibilities

• Orchestrate data workflows and data movement using Azure Data Factory or Azure Data Bricks.
• Build, test, and run data assets based on tasks and user stories from Azure DevOps.
• Handle break fixes and participate in a rotational on-call schedule, monitoring jobs and ETL pipelines.
• Design and implement CI/CD pipelines for software development teams.
• Contribute to producing and maintaining operational and architectural documentation.

Required Skills

• 10+ years of professional experience in data engineering.
• Expertise in building end-to-end ETL cycles using Azure Data Factory (ADF) or Azure Data Bricks (ADB).
• Proficiency with PySpark and Python for data transformation.
• Experience handling sources like REST APIs and databases (Oracle, etc.) and targeting Azure SQL or Azure SQL Database.
• Familiarity with ADLS, SQL, and SQL Server.
• Working knowledge of Safe/Scrum Agile frameworks.
• Experience with Azure DevOps for CI/CD processes.

Key Skills

Education

Any Graduate