Inducing novelty in generated descriptions

Prompting
1

Hi guys,

I wanted to understand what the logit_bias parameter in Completion.create means and if I can use it in a particular way.

I referenced the documentation and if I understand it correctly, providing a -100 value to a token in the logit_bias dict will result in drastically decreased likelihood of selection of that token during generation of a new sentence, right?

Then I have a couple of questions based on this:

  • I want to create 5 unique product titles. Assuming I know how to nicely do prompt engineering, can I successively add the tokens of generated titles after every run to the logit_bias dictionary before getting the next generation? For eg. I generated description1 (d1), then add tokens corresponding to d1 to the logit bias for getting d2. Then add d1 tokens and d2 tokens to the logit bias for getting d3 and so on.
  • Does passing the logit-bias dictionary not work for stopwords like this, a, an, the etc. because I tried adding these to the dictionary but they seem to appear irrespective of specifying them in the generation; which may be a good thing as well. I just want to know if there’s a set of stop words for which logit-bias is not applied…

Thanks,
Vishal!

2 Likes
3

Yes, great idea. Just careful about how the text is tokenized - most common words are prefixed with a space.

You can also request logprobs=7 which gives you 7 alternatives at each token position. Then for the next request you can force the first token to be one of the less likely ones.

2 Likes
4

Hey @boris

Thanks for the prompt response.

Could you please elaborate on the logprobs parameter? I didn’t get it. An example would be highly appreciated.

Thanks,
Vishal!

5

documentation for logprobs: OpenAI API

example which uses logprobs: openai-python/finetuning-classification.ipynb at main · openai/openai-python · GitHub

2 Likes