If you've ever read John Moehrke's blog you know he talks a lot about provenance. One of the challenges I've been faced with recently is the need to track the provenance of an AI output. The things I need to know are:
- What was the prompt that produced this, so I can reproduce it again at a later date (this can be used to detect drift and change).
- What files or inputs were used to produce it. These should be URIs where feasible so that I can get to the content, not just summary descriptions found in the prompt, and when tied to an HTTP address, even the date of retrieval for the content used.
- What agent generated it. Like many, I'm using multiple agents. What version of the agent is also useful.
- What LLM and version were used. Many agents support multiple models.
The first of these is actually not too difficult to identify. I wanted this to be a URI as well, but right now there is no URI for a prompt, so I made up a syntax. It follows and can also be found here. I'm interested in getting community feedback before I spend any time trying to submit this to IETF.
0 comments:
Post a Comment