Hello!
I’m uploading 3 to 6 files pdf files to a thread (thread vector store) I’m running with an assistant. I find that sometimes it seems that the file contents haven’t been absorbed fully by the assistant as he replies without knowing the contents of them. This seems a bit random so I assume it’s linked to load on OpenAI end. I’m not seeing a way in the API to check if the file contents have been absorbed or somehow waiting - how are you managing this so it’s reliable?
Thanks!
After doing the upload, we poll on the vectorStores.files.retrieve for that file and check the status and wait until it is “completed”. I believe at that point, the information will be available for use.
Thanks for the suggestion kduffie. Are you experiencing 100% success rate?
We’re already doing that and waiting for “completed” but it doesn’t seem to be correlated with the AI model actually having consumed/trained on the data. Our files per thread are up to 6 PDFs with combined perhaps 20 pages of text.
Other thing we tried was waiting 20 secs before running the thread - that doesn’t change anything.
The only thing we tested and seemed to actually improve the result was to run the thread with about 50/50 chance the AI will answer knowing the data of the files or not. And then immediately run it again with just “Try this again.” and it raises the successful runs up to ~95%. We still get occasional fail and we’ve consumed about 15% more tokens.
It feels to me like that even though the file upload is completed, the AI model hasn’t consumed/trained on them and there is no way to poll if that’s done before running the thread.
I’ve tried this in the assistant playground and observe the same behavior there which indicates to me that the OpenAI API is not managing this correctly!
I’m hoping someone has faced this issue and found a way to manage it.
First, there are others who have more expertise than me. So I hope you may get an answer from an insider who knows the actual code.
Having said that…
In our use case, the files being uploaded are part of a big repository – often thousands of documents. So when a new one goes in, it is rarely obvious that it is missing immediately after upload, so I can’t give you any definitive answer about that. I do note that the polling time required waiting for a file to be “completed” varies a lot – and is sometimes minutes rather than seconds. That makes me suspect that they are reporting that at least one major stage is completed, but perhaps there could be caching or other things at play.
Your idea to “try again” gave me an idea. If this is RAG content, then you could easily write a “test” that uses a different specific instruction to “Search for ____” using some signature that you add into the upload so that it will be unique among the content. Recall that you can find out via the run API exactly which documents were returned by file_search, so that is a definitive way to find out if the file is available via file_search.
It’s our experience as well that the time waiting for a file to be completed varies a lot.
Your idea of using the search as a mechanic to check if the file is ready interesting. We’ll look into that, thanks!