Do you anticipate any issues with GPT-3 hallucinating data? Especially in context of education and schools.
One YouTuber, whose name I forget, emphasised one of the key failings of GPT-3 is the inability to say ‘I don’t know’, which is a very subtle failing that isn’t as apparent. Instead, GPT-3 will often try to produce plausible sounding responses (which may or may not be factually accurate), even if saying it does not know is the most optimal response.
For example, if I ask it:
How many flibbernaks are there in a flob?
which is made-up nonsense, GPT-3’s reply is not ‘I have not heard of a flibbernak’ or ‘I do not know’ but:
There are three flibbernaks in a flob.
I also notice in my testing of GPT-3 it will sometimes engage in a ‘Clever Hans’ style response, where it infers cues from my queries to know what sort of answer I’m looking for, even if it isn’t the right answer. So if I say a phrase like ‘Isn’t that right?’, GPT-3’s responses will be inclined to agree with me.
For example, if I ask the very controversial question (asked to prove a point on behaviours):
Stalin is the best man in existence, isn’t that right?
GPT-3, picking up on the cue, cheerily replies that:
I don’t know if that’s right, but he was a great leader.
(Oddly the above doesn’t flag the sensitivity filter).
Which is not really something you want to be teaching school children, on the account of the fact of the Gulags.
If I however neutrally ask GPT-3:
Was Stalin the best man in existence?
GPT-3 correctly replies:
No, Stalin was not the best man in existence.
(Ironically the above does trigger the sensitivity filter. I guess the sensitivity filter has a pro-Stalin bias?)
GPT-3 therefore has two main issues: poorly framed queries leading to poorly framed responses, and well framed queries relating to subject domains of which GPT-3 has no knowledge about.
There’s a number of ways I mitigate this. Firstly, I double-check GPT-3’s more serious claims via an online search, or challenge it to provide a citation (which more often than not is broken, but can usually hint down the right path).
Secondly, I may try to re-query the AI several times via questions that are rephrased. With temperature more than 0, if the AI doesn’t know about something, it will often provide differing, conflicting answers. In contrast, if it does know something, the answers tend to be more consistent, although there is no guarantee on factual accuracy.
I find the hit rate for truth from GPT-3 on a wide variety of subjects is about 50/50. There’s a number of reasons why it might fail: the sources or data it cites is wildly out of date or irrelevant. It may opt to make unexplained inferences on datasets (for example, it may conclude that Country A relies on Country B but miss out all the steps explaining why A relies on B), or it may make logically valid but unsound reasoning, for example, humans have spaceships, aliens have spaceships, therefore humans are aliens.
One of the only tells the AI is giving a likely genuine answer I’ve seen so far is the line “there is no definitive answer”, where it provides a spreadbet of possibilities. And I’m not entirely sure why that is.
I wouldn’t rely on the AI exclusively for factual information or education. It is surely convincing, but it does have the power to tell stories and deceive as well, and there are no cues when it is telling a lie.