How to prevent GPT-3 from inventing things that don't exist?

Prompting
1

I’m trying to get GPT-3 to recommend a list of books based on a description.
The problem is, GPT-3 always tries to force a list of 10 books, even though the prompt says that less than 10 books is also ok.
This causes GPT-3 to invent books that don’t actually exist. My attempts to fix the prompt didn’t work so far:

const completion = await openai.createCompletion({
    model: 'text-davinci-003',
    prompt: `Book bot: "I am a highly intelligent book recommendation bot. 
    I recommend what books to buy next based on your description.
    I am helpful, friendly, and truthful."
    I only recommend books that actually exist and can be bought today. 
    I am not creative and I don't invent any non-existing book titles or authors. 
    If there are less than 10 books matching your description, I will only respond with these.
    If there are no books matching your description, or if your description is nonsense, I respond with "Nothing found". 

    Requester: "Book bot, please recommend up to 10 (or less) books based on the following description: '${query}'".

    Book bot: "I can recommend the following books based on your description: 
    1.`,
    temperature: 0,
    max_tokens: 400,
    presence_penalty: 0,
    frequency_penalty: 0,
  });
2

Truth is hard. Another approach might be to build a (large) database of real books for the bot to select from.

3

Thanks for the tip but creating this database of books is not feasible.

Maybe someone knows how to get rid of these fake entries?

4

Maybe try using a prompt that implies real books such as “for sale on Amazon” or “reviewed by the NY Times” or “appeared on a bestseller list.” Alternatively perhaps you can fine-time by providing a few hundered Q&A pairs where all of the books are real books.

1 Like
5

Thank you! Those are good ideas!

I actually tried the “available on Amazon” approach already and it didn’t work.

But I’m gonna try your other idea of preparing a few example prompts where some of them only contain less than 10 results (GPT-3 always tries to force a list of 10).

The only downside is that this will increase the costs because of the larger prompt.

Anyway, thank you very much for helping!