Description

Job Description

Key Responsibilities:

• Design and build end-to-end data pipelines and ingestion workflows using Azure Data Factory and Microsoft Fabric, ensuring scalability, observability, and data quality at every stage.
• Develop and maintain serverless automation and event-driven integrations using Azure Function Apps and Logic Apps, including orchestration flows that connect AI services to data pipelines.
• Own infrastructure-as-code practices using Terraform to provision and manage cloud resources consistently across environments.
• Drive CI/CD pipeline development and deployment automation through GitHub Actions and Harness, ensuring fast, safe, and repeatable deployments across both data and AI workloads.
• Integrate Azure AI Services and Azure OpenAI into data workflows - including document intelligence, embeddings, semantic search, and AI-assisted data enrichment - enabling the platform to support intelligent applications and copilot-style experiences.
• Architect/Manage analytical workloads within Azure Synapse Analytics, including Lakehouse patterns, dedicated SQL pools, and Spark-based transformations.
• Support cross-cloud strategies by applying AWS knowledge to maintain contingency pipelines and hybrid architecture patterns.
• Maintain continuity of existing pipelines and maintenance work.

Qualifications & Requirements:

• Around 10 years of experience in data engineering with a strong focus on cloud-native solutions.
• Deep hands-on expertise with the Microsoft Azure data stack, including Azure Data Factory, Microsoft Fabric, Synapse Analytics, Logic Apps, and Function Apps.
• Proven experience with infrastructure-as-code using Terraform, and CI/CD platforms such as GitHub Actions and Harness.
• Strong proficiency in SQL and at least one data-oriented programming language, with Python preferred given its centrality to AI and ML integration work.
• Demonstrated experience integrating Azure AI Services or Azure OpenAI into data pipelines or platform workflows, including working with REST-based AI APIs, prompt engineering, embeddings, or AI-assisted data processing.
• Practical experience publishing data to SharePoint Lists from ADF pipelines, including familiarity with SharePoint Online REST APIs and OAuth authentication flows.
• Working knowledge of AWS services - such as S3, Glue, or Lambda - sufficient to design and operate contingency or hybrid workflows

Key Skills

Education

Any Graduate