Are there plans for min_tokens parameter?

Hi there,

This is an incredible tool. I just had a small query about it. While there is a max_tokens parameter, there does not seem to be a min_tokens parameter. Are there plans to introduce this, or is there any workaround?

Just for context, I am building a children’s bedtime story generator as part of a college project and would like a minimum length response of 100 tokens. I realise I could add something to this effect in the prompt (“Finish this bedtime story in 400 words or more”), but I have found that the best story results come from just providing a title and stem paragraph with no instructions.

Many thanks,

Dan

2 Likes

Welcome to the OpenAI community @dananthonyobrien!

That is an interesting idea, however I suspect that it may produce undesirable completions at times as well. You’d essentially be forcing the models to keep generating the completion, which could cause poor results whereas simply advising them that producing a story with X number of words in it would be less constraining for the models.

Think of that like having to write an essay with X amount of words in it. While some students may be able to produce an essay without having to worry about the word limit, others may struggle towards the end which would cause them to have to resort to finding ways to produce more words but instead of writing fresh content they may instead start reiterating ideas or use filler words to extend the length of their sentences despite the impact that may have on the overall quality of their essay stemming from being forced to produce X amount of words instead of being given the option to write as much as they feel they can.

I think the above explanation is correct for describing the possible obstacles the model could encounter if having that constraint, however I could certainly be wrong here so I would not assume my statement is correct.

You could try re-engineering the prompt in ways you hadn’t tried yet to see if you get more desirable completions. If you haven’t yet, you could try telling the model something along the lines that it is a “creative storywriter for children and their audience enjoys stories of this length the most.” It’s really all about trying different things until you get the results you want, and you’ll find that adding more detail to the prompt tends to help more than not.

If you are still struggling to get the model to generate the desired results you want though, definitely reach out here on the forum and we’ll be able to provide you with what helped improve the quality of the models’ completions through our own personal experiences trying to get the results we wanted. We’re more than happy to help!

2 Likes

For the most part I noticed that GPT-3 is almost completely oblivious to the number of words it is generating or even the general length of its completion. I have told it specifically to generate one sentence and have seen it generate a rather long paragraph. Perhaps having a suggested length parameter so GPT-3 has some freedom but penalizes tokens that exceed the suggested length and reward tokens for getting closer? Just a suggestion but it would be greatly helpful.