I have created an Assistant with the Assistants API, then created a Vector Store and assigned the Assistant to the store. I chose the gpt-3-turbo model when creating the assistant.
Then I uploaded some files about a certain topic (a Business) and added them to the vector store. After the files were ready, I started asking the Assistant questions and got very good answers back, based to the file_search tool.
Then I changed the Assistants’s model to gpt-4-turbo and it seems like it has lost all the knowledge from the uploaded Vector Store files. Changing the model back to gpt-3-turbo makes it work again.
Do I have to remove and re-add the files to the Vector Storge again when changing model in the Assistant?
3 Likes
Seems like this is happening when you change the Assistant’s model and don’t add model to the RunCreateParamsBaseStream when creating a new Run with that Assistant.
So basically, your Assistant “forgets” the vector store content, unless you add model to the parameters when you create a new run.
Same thing with temperature, if you do not add the temperature in the parameters when you create a run, it seems to ignore the temperature setting of the assistant.
Not sure if some of this is intended or not, but seems like a bug.
This seems to be an issue again, happens regardless of which model I pass into the RunCreateParamsBaseStream (except the “original” model the assistant had when creating the vector store files).
Changing the Assistant’s model, after uploading files and adding them to the vector store, makes the assistant completely “forget” all the content in the vector store.
It is also an issue when testing in the OpenAI playground.
Changing the model back to the original, makes the assistant aware of the vector store content again.
In this particular case, the assistant had model gpt-3.5-turbo when uploading the files, switching to gpt-4-turbo breaks the assistant.