Hello,
I’m using the built-in Retrieval feature. I uploaded a single file as a knowledge base, which has about 1200 words (so it’s not very big). I’m facing the following issue: The model answers the first question of a thread very accurately, but for the second question in the same thread, it starts to give incorrect answers (hallucinating). For example:
Thread 1:
Thread 2:
This is reproducible for all kind of questions.
Does the model not use the knowledge base for the second question too? I’ve been trying this with gpt-4-turbo-preview.
Any idea why this is happening?
sps
March 10, 2024, 3:28pm
2
Hi @LorenzHW
Are you uploading file with message on the thread or is it part of the assistant’s configuration?
1 Like
@sps It is part of the configuration of the assistant.
However, I did an experiment: when I upload the file also with the message it seems to work. E.g my model doesn’t hallucinate on the second query.
I am not really satisfied with this solution, because takes more time and is more expensive. So any insights might be helpful
I noticed that when I run it in the Playground, ‘Retrieval’ runs only on the first query of a thread. Never on the second.
1 Like
sps
March 10, 2024, 4:56pm
4
I created a simple assistant on playground with a .txt file and had similar observations as yours.
Then I appended “Refer to the provided txt file for knowledge.” to the instruction in it’s config and it’s running retrieval for every message.
Hence, you have to instruct the assistant to refer to the provided file.
1 Like
Hmm… I added it to my instructions as well but it doesn’t seem to help. I will keep experimenting.
1 Like
