Description

You will design, develop, and deploy data pipelines that leverage agentic AI to solve real-world problems.

Responsibilities

• Design and develop data pipelines and robust data flows to handle complex interactions between AI agents and data sources.
• Build data architecture, including databases and data lakes, to support agentic systems and engineering tasks.
• Develop and manage ELT processes to move data from source systems to analytical platforms.
• Implement data pipelines that facilitate feedback loops for human-in-the-loop systems.
• Work with vector databases to store and retrieve embeddings efficiently.
• Collaborate with data scientists to preprocess data, train models, and integrate AI into applications.

Required Skills

• 3+ years of experience developing Python scripts.
• 3+ years of experience training LLMs with structured and unstructured datasets.
• 2+ years of experience working with GIS spatial data for geo-location and topology tasks.
• Experience with Spark, Databricks, and GraphDB.
• Experience with Azure services including Azure Data Lake, Azure Databricks, and Azure OpenAI models.
• Proficiency in vector databases and embedding models for retrieval tasks.
• Expertise in integrating with AI agent frameworks.
• Knowledge of data partitioning schemes and data conflation.
• Understanding of core machine learning concepts, algorithms, and statistical analysis.

Preferred Skills

• Experience with Department of Transportation Data Domains.
• Experience developing AI Composite Agentic Solutions for forecasting and what-if analysis.

Key Skills

Education

Any Graduate