Ha ha don’t make me cry.
I agree that some sort of validation is needed to make sure the quality of the text is good. I wrote up the sequence that’s needed here yesterday. I guess this is the price we pay for having Langchain RAG pipelines abstracted away?
So upload file, then use get-content to pass it through a completions endpoint for validation/summarization, if good attach file to the message and then pass the validation/summary to the assistant as additional instructions? That will at least let you know whether the file checks out.
Having a smallish documents tokens mixed in with a full-sized or multiple vector stores is still a problem, I don’t see any way around disconnecting the datastores for the run then reattaching, but so much can go wrong there, having different agent would be better I think. Not really sure this is Assistants problem per se as I imagine this affects any RAG flow.
Thread expiration is an issue that we can design around by our resume thread ux. We can tell which user messages had an attachment, and we can query the thread’s attached datastore by file ID to see if it’s still there, at which point we can error to the user accordingly or even re-upload the file from our custom data layer.
We may even modify the file upload flow to give users option to choose between a default thread upload or attaching to the assistants default vector store.