You will build and manage the data infrastructure required for autonomous agentic systems.
Responsibilities
- Design and develop data pipelines specifically for agentic systems.
- Build data flows to manage complex interactions between AI agents and various data sources.
- Train LLMs using both structured and unstructured datasets.
- Integrate GIS spatial data into existing workflows.
Required Skills
- 5+ years of experience in data pipeline design and development.
- Experience training LLMs with structured and unstructured data.
- Proficiency with Apache Spark.
- Experience with Azure Databricks.
- Experience working with GIS spatial data.
- Background in big data frameworks.
- Bachelor's degree or equivalent professional work experience.
Preferred Skills
- Experience with GraphDB.
- Knowledge of data partitioning and data conflation.