Description

You will build and manage the data architecture and pipelines required to power agentic AI systems.

Responsibilities

• Design and develop data pipelines for agentic systems to handle complex interactions between AI agents and data sources.
• Build data architecture, including databases and data lakes, to support ELT processes and analytical platforms.
• Implement pipelines that facilitate feedback loops for human-in-the-loop systems.
• Work with vector databases to store and retrieve embeddings efficiently.
• Collaborate with data scientists to preprocess data, train models, and integrate AI into applications.
• Perform statistical analysis to create data formats from multiple sources and optimize storage retrieval.

Required Skills

• 5+ years of experience in data engineering and machine learning.
• Strong programming skills in Python and experience with AI/ML frameworks.
• Proficiency with Big Data frameworks including Spark and Databricks.
• Experience with Azure Data Lake, Azure Databricks, and Azure Blob Storage.
• Hands-on experience training and fine-tuning LLMs with structured and unstructured datasets.
• Expertise in integrating with AI agent frameworks and using vector databases/embedding models.
• Knowledge of Graph DB and core machine learning algorithms.
• Experience with Azure AI services, including Azure OpenAI, Azure AI Search, and Azure Machine Learning.
• Experience with GIS spatial data, including geolocating datasets and topology correlation.
• Experience with Department of Transportation Data Domains for predictive and what-if analysis.
• Bachelor's or master's degree in Computer Science, AI, or Data Science.

Key Skills

Education

Bachelor's or master's degree in computer science, AI, Data Science