Prevent hallucination with gpt-3.5-turbo

API
5

Okay, after some test after @nicole_n said here

When I use like this:

completion = openai.ChatCompletion.create(
       model="gpt-3.5-turbo",
       messages=[{"role": "system", "content": """Answer the question as truthfully as possible, and if you're unsure of the answer, say "Sorry, I don't know".\n\n{context}}"""},
                          {"role": "user", "content": f"{question}"}]
)

It will reply
Sorry, I don't know what the question is. Could you please provide more context or clarify your question?

The first 4 words are correct, but it seems that gpt-3.5-turbo has lengthen the reply.
Then I’ve notice that my configuration is:

temperature= 0.0,
max_tokens= 300,

I guess because of the max tokens is far above reply text tokens so the model will auto lengthen the reply. So please add more words to “Sorry, I don’t know” or lower the max_tokens.
By doing this, I think the model will reply the exact text you want.

And another thing that I discovered is that if your question and the text (“Sorry, I don’t know”) is in different language. It will not reply the exact text you want too.
About this issue I didn’t figure how to deal with multiple language. Feel free to discuss below.

1 Like