In general, the answers I receive from gpt-3.5-turbo-16k have ranged from middling to non-existent. I stopped using it for this reason on the important legal document store that my app maintains.
However, for support documentation searches, I am using it simply because of the cost. My documentation focuses on how to use the the query system (a conversational chat search app using OpenAI). I constantly make the point in this documentation that while the AI may seem human and knowledgeable, it is neither of those things. Based on the ideas expressed in most of my current documentation, this must be the absolutely best response I’ve ever received from gpt-3.5-turbo-16k:
What are the support documents like? Do you have an example of one I can test?
How are you including them?
If I take the question you have posted as is, then you have not included any documents, simply some text describing the name of the documents, possibly in hyperlink format, but that will still not be read by the model, are you including the file content with each API call? if so do they go over the 16k limit?
Thanks, I appreciate it, but at this point, I’m pretty convinced that gpt-3.5-turbo-16k cannot be used for this particular application. If it’s working for you, that’s great, but on the documents I’m working with, I’m getting mediocre to bad responses pretty consistently.
Here is an example of non-existent, where the answer is in the document gpt-3.5-turbo is reading, but it fails to see it. Again, consistently, in the API and the playground. GPT-4 and even Claude both respond with the correct information with no changes whatsoever to prompt or format.
@Foxabilo did assist me in finding the right prompt, but the problem is that I can’t expect end-users to figure this out on their own.
Now, it could just very well be the way my data is formatted, I just don’t know. The only thing I know is that the answers, relative to the to the other models, are consistently poor.
Actually, there’s not a problem. The model responded correctly which is the whole point of why I posted this topic. It responded precisely as it should have, given the context of the available documents:
All pretty dry, uninteresting stuff. Since this is a public facing page, I decided to use gpt-3.5-turbo-16k, because I have to. I was just doing some tests to see what it would say, and this was one of them.