Description

You will design, build, and maintain scalable data pipelines to support Gen AI applications.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines using Azure Data Factory, Synapse Pipelines, and Databricks.
• Ingest, transform, and curate structured and unstructured data from diverse sources for AI/ML consumption.
• Enable data access and preparation for Azure AI services including Azure Machine Learning, Azure OpenAI, and Cognitive Services.
• Collaborate with data scientists and ML engineers to operationalize AI models using MLOps and GenAIOps practices.
• Implement CI/CD pipelines for data workflows using Azure DevOps or GitHub Actions.

Required Skills

• 10+ years of experience in data engineering, cloud platforms, and big data technologies.
• Strong proficiency in Azure services: Data Factory, Synapse, Data Lake, Azure SQL, Cosmos DB.
• Experience with Python, Spark, SQL, and distributed data processing.
• Familiarity with AI/ML workflows and tools such as Azure ML, MLflow, and OpenAI APIs.
• Understanding of data governance, security, and compliance in cloud environments.
• Experience supporting RAG (Retrieval-Augmented Generation) patterns.

Key Skills

Education

Any Gradute