In general, the answers I receive from gpt-3.5-turbo-16k have ranged from middling to non-existent. I stopped using it for this reason on the important legal document store that my app maintains.
However, for support documentation searches, I am using it simply because of the cost. My documentation focuses on how to use the the query system (a conversational chat search app using OpenAI). I constantly make the point in this documentation that while the AI may seem human and knowledgeable, it is neither of those things. Based on the ideas expressed in most of my current documentation, this must be the absolutely best response I’ve ever received from gpt-3.5-turbo-16k:
I would like to help but I’m confused, so you don’t use gpt3.5?
Based on what parameters did you arbitrate this non-existent part? I ask this because I’m using it in practically 70-8/% of the current scenario of my applications.
What are the support documents like? Do you have an example of one I can test?
How are you including them?
If I take the question you have posted as is, then you have not included any documents, simply some text describing the name of the documents, possibly in hyperlink format, but that will still not be read by the model, are you including the file content with each API call? if so do they go over the 16k limit?
Thanks, I appreciate it, but at this point, I’m pretty convinced that gpt-3.5-turbo-16k cannot be used for this particular application. If it’s working for you, that’s great, but on the documents I’m working with, I’m getting mediocre to bad responses pretty consistently.
Here is an example of non-existent, where the answer is in the document gpt-3.5-turbo is reading, but it fails to see it. Again, consistently, in the API and the playground. GPT-4 and even Claude both respond with the correct information with no changes whatsoever to prompt or format.
@Foxalabs did assist me in finding the right prompt, but the problem is that I can’t expect end-users to figure this out on their own.
Now, it could just very well be the way my data is formatted, I just don’t know. The only thing I know is that the answers, relative to the to the other models, are consistently poor.
You can see from the image I posted that the total tokens for question and query (and standalone and concepts) is 2322. There were only 3 context documents returned.
It’s just a quick little query screen I put together using the API I created for the app, designed to search only the support documents for the site.
Because a lot of the site content is regulatory in nature, a lot of the support documentation always stresses the importance of checking the returned context documents.
And that’s why I call this the “best answer” I have ever received, because gpt-3.5-turbo-16k finally seemed to get it right the first time out!
Actually, there’s not a problem. The model responded correctly which is the whole point of why I posted this topic. It responded precisely as it should have, given the context of the available documents:
All pretty dry, uninteresting stuff. Since this is a public facing page, I decided to use gpt-3.5-turbo-16k, because I have to. I was just doing some tests to see what it would say, and this was one of them.
