Why do the models hallucinate?

API
1

Hi ,
I was going through this deepAI learning course in association with openAI:
learn.deeplearning.ai / courses / chatgpt-prompt-eng / lesson / 2 / guidelines
This is what is mentioned in this video:

image
image956×460 50.9 KB

It is told that Boie is a real company, the product name is not real and the response the model gives to the prompt is incorrect.

Code:

client = openai.OpenAI()

def get_completion(prompt, model=“gpt-3.5-turbo”):
messages = [{“role”: “user”, “content”: prompt}]
response = client.chat.completions.create(
model=model,
messages=messages,
temperature=0
)
return response.choices[0].message.content

prompt = f"“”
Tell me about AeroGlide UltraSlim Smart Toothbrush by Boie
“”"
response = get_completion(prompt)
print(response)

The tutor says that the response is incorrect because Boie is a real company but the product is not.

Question:

  1. Why does it happen that the model ‘invented’ as incorrect answer. The LLM’s are an example of supervised learning. So, if we did not teach the model about this product which does not exist, why did it get an answer ?

  2. So, what is the level of confidence the users of GPT have when using these models? As a user, I would always be in a dilemma whether the model returned me correct answer or not.

2

It’s a prompting issue in the vast majority of cases. Is the model allowed to say, “I don’t know?” . It’s often the case that “Yes” or “No” is just a more likely output on a given prompt, and the model will then run with that.

IMO LLMs shouldn’t be used as search engines.

If you pay close attention to the limitations of these models, you can get reliability rates in the high 90’s. But the output will never be guaranteed to be accurate. This is simply a different paradigm. If you want classical outcomes, you need to use classical algorithms.

3 Likes
3

I never pressed send on this draft, so I’ll finish my thought.

To start, it is good to take a few steps back from what you think of now as a “chatbot” that is an entity you interact with, and look at the technology that powers it.

Transformer AI is a one-directional completion that generates the next likely token in a sequence, and then continues this process iteratively. It is able to do this by being pretrained on a large corpus of text with a reward model.

So, take indescribable amounts of text, from internet posts, press releases, philosophy books, and what have you, and the AI begins being built (and rewarded on) observations of the correct sequences of words, to where it can predict the next word, phrase, sentence, paragraph with high accuracy.

That is transformed into a chatbot by simulating a document where we see two “people” interacting. As the AI has been more trained by OpenAI and uses a special chat format of input, you need less of “here is an example of a conversation between a human and a smart AI”.

Now: The AI is going to write whatever response is plausible before it has been trained to “chat”. If you ask it for scientific papers about the grooming of cats, it will make up links. If you ask it to summarize an article where the URL has hints that aliens landed in Tokyo, the AI will infer the likely response, a fact-like recounting of a whole story about aliens.

To have anti-hallucination, and to deny things the AI cannot do or know takes extra extended supervised training, not simply the training on vast text. Adding training with responses like “I can’t access links on the internet” or “that looks like a fake link you made up”.

This “truth” training takes coverage of many scenarios. For example, ask an AI about the quality of your accent, and it could still give fantastical advice on how to pronounce words even though it can’t hear them, just because that looks like the kind of response that would be appropriate for such an input.

“Confidence” is a strange thing to discuss. The AI is just generating one word at a time, so the only thing it is confident about is the next word to produce from the training weights.

2 Likes