Description

You will build and maintain data infrastructure supporting AI and LLM workflows.

This role is hybrid.

Responsibilities

• Implement LLM-based workflows, RAG pipelines, and agentic orchestration using LangChain or CrewAI.
• Design, develop, and maintain scalable data pipelines using Python and Azure services.
• Containerize and deploy solutions using Docker and Kubernetes within cloud-native architectures.
• Ensure data quality, observability, and compliance across all deployed data systems.
• Collaborate with data scientists to translate business requirements into technical data solutions.

Required Skills

• 6-8 years of experience in data engineering with a strong focus on Python and Azure.
• Proficiency in advanced ETL/ELT processes for structured and unstructured data.
• Extensive experience with Azure cloud services: Azure Data Factory, Azure Databricks, Azure SQL Database.
• Expertise in Python for data processing and automation.
• Working knowledge of CI/CD principles and tools such as Jenkins or GitLab CI/CD.
• Experience with containerization and orchestration tools like Docker and Kubernetes.
• Familiarity with Big Data tools including Spark, Hadoop, and Kafka.
• Strong command of SQL and experience with NoSQL databases.
• Experience with generative AI concepts, including prompt engineering and LLM fine-tuning.

Key Skills

Education

Bachelor's or Master's degrees