How to encourage/force instruct models to choose among options

Prompt Assistance
#1

Hi, has anyone had any luck at encouraging/forcing the instruct models to choose from among a set of items? For example, something along these lines:

Generate a recipe that uses one of [blueberries, raspberries, blackberries, strawberries, chokeberries].

Desired responses might include:

  • a recipe that uses exactly one of the terms in the set
  • a recipe that uses one or more of the terms in the set
  • a recipe that uses terms similar to those in the set
2 Likes
#2

Few shot is your best example. The problem with this kind of task is that it requires working memory, playing with multiple options. Gpt-3 doesn’t have that capacity.

#4

Just a general challenge, for any list.

1 Like
#5

Fine-tuning would definitely help, but few-shot would be the next best alternative. You could also try using logit bias.

The easiest way would be to modulate the input to the API, rather than use the API to try to only pick specific parts of the input for a generation (e.g. instead of saying “Generate a recipe that uses one of [blueberries, raspberries, blackberries, strawberries, chokeberries].” you could say “Generate a recipe that uses random_selection([blueberries, raspberries, blackberries, strawberries, chokeberries].)”

To add to that, by random_selection(), I mean that you’d program that functionality in, not ask the API to randomly select from the list.

6 Likes
#6

@joey That’s super interesting. Is there a list of commands that can be used, or everything works if it’s in that format?
I have a similar problem, where I need the model to generate a reply for a natural language prompt in a particular format to later connect to an API.