I am working on a chat bot for Airbnb guests to ask questions about their Airbnb and local businesses. I have files for it to access this info (md for things like house rules, and json for businesses). I can confirm that file attachment and such are all correct, retrieval is on, etc. In testing, I found that in a new thread, it can be unpredictable when it will choose to access files. Before it does so, it’s pretty erratic, and will hallucinate, giving random restaurants/businesses. In my instructions I tried telling it to always refer to the files it has access to, and not to make any assumptions, but it doesn’t seem to work consistently.
Just to note, prior to implementing streaming (nodejs) it seems to retrieve immediately and was much more predictable. It’s also predictable in the playground.
If I ask a question like “Can you suggest a place for dinner?” for some reason it won’t try retrieval. Although, if I ask a more specific question, like “Can you suggest a pizza place?” it will find the info for the pizza places in my json. My assistant really requires retrieval to be used immediately, otherwise it’s extremely unpredictable.
So I suppose my question is, how can I get it to retrieving data from files more reliably? I don’t know the mechanism it’s using to make this decision so it’s a bit of a guessing game. i.e. Do I need more metadata in the json to let it know when to use the file? It would be great if files had a “description” field like functions did, or used another method to understand when to retrieve data.
I know that I can add files to threads as well, but in some of my earlier tests trying that it seemed like I had the same issue. It was aware of a file, but it didn’t necessarily analyze it. I could be wrong about this, since I haven’t tested that as thoroughly.
This screen shot is just a test, but I just asked my assistant what files it has access to in the beginning of a thread, but it just doesn’t know what they are yet.
