Description

You will build and maintain data pipelines supporting AI/ML workloads.

This role is hybrid.

Responsibilities

• Manage and monitor end-to-end data pipelines and jobs.
• Maintain and manage Retrieval Augmented Generation (RAG) indexes.
• Integrate and onboard new potential data sources into the ecosystem.
• Build and consume services using Rest APIs.

Required Skills

• 5+ years of experience in Data Engineering or a related capacity.
• Strong proficiency in Python and PySpark.
• Expertise in SQL and data warehousing concepts, including dimensional data modelling.
• Hands-on experience with Azure DataBricks and the Azure cloud data platform.
• Familiarity with Delta Lake architecture and the Medallion pattern.
• Experience leveraging Azure services like Azure Data Factory, Synapse Analytics, and Event Hubs.
• Proficiency with CI/CD pipelines and DataOps practices.
• Experience with Azure DevOps.

Preferred Skills

• Any Graduate.

Key Skills

Education

Any Graduate