There are many different reports of this happening.
- Assistant API always return empty annotations
- Mapping assistants API annotations back to the location in the source file
- Assistant message with annotations and source tags - #4 by JayArrBee
- What are 【N†source】in OpenAI's Assistant API response?
- Assistants API fails to use Retrieval tool as effectively as GPTs
- How can I access file_citation?
From our experience:
- Assistants are more reliable with annotations when these come from assistant-linked files rather than thread (message) files.
- Asking for annotations, sources seems to improve likelyhood of them being included.
- Nonetheless it seems to very very often include references without the related sources. (I rarely see more than 2 correct annotations)
- The ‘text’ inside annotations is supposed to contain the source text to replace but often contains other, unrelated text. Thus, the snippet suggested by OpenAI in their documentation tents to break the answer in unexpected ways (as you’d be replacing random texts with references)
We would appreciate any additional information the OpenAI team can provide on how to make annotations work consistenly or at lease more reliably.
Thanks