Why language models hallucinate [OpenAI Research Paper]

Community
1

OpenAI just published a new paper on hallucinations in Language Models.

tldr: “Our new research paper ⁠(opens in a new window) argues that language models hallucinate because standard training and evaluation procedures reward guessing over acknowledging uncertainty.”

Wanted to start a new topic to dicuss this paper, and also some of the ways the community is working to reduce hallucinations in outputs, whether using the API or on ChatGPT.

I’ll start by saying that one common way I’ve seen discussed in general is to give the model an ‘out’, by allowing it to skip answering a question if it doesn’t know the answer, by specifiying in the prompt: “If you don’t know the answer, say so” (or some variation of this). This has yielded mixed results. What other ways have you been using to detect and reduce occurences of halucinations?

Reference Links:
Webpage: https://openai.com/index/why-language-models-hallucinate/
Paper: https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4aaa5/why-language-models-hallucinate.pdf

3 Likes
2

you want it to not hallucinate?
don’t make it role play, the act of doing so automatically creates such outs for ‘hallucinations’ or guessing.

in your context file of how it should behave it needs to demand to be honest always, to not roleplay, to not simulate, and to state specifically if information is not available.

problem solved.

I’ve been running like this for months and haven’t had a hitch since…

it’s that role-playing thing people get it to do that opens the door the most in my understanding.

3 Likes
3

@jai Thank you for sharing, it’s an interesting piece by OpenAI. You can ask it to avoid making things up and mostly it will follow that though there tends to be some drift.

If you’re interested, I made 2 custom GPTs to help ID hallucinations in the responses. You’re welcome to try them out here:

Diagnostics Mode GPT

Hallucination Check

Feel free to suggest feedback, thoughts, etc. Good luck!

–Gary

1 Like