Fixed word count

[raoul]

Hi All,

I looked at the sample below to try to get a fixed length word count:

How to get instruct series to limit the output length?

Using Davinci instruct. It doesn’t seem to obey instructions very well, like “The paragraph should be 2 sentences long”…

Since it’s a year old, are there any better solution then recursively calling and asking it to add words?

Thanks,

[lmccallum]

Check out the latest models here: OpenAI API. You can include instructions in the prompt that “answers should be 2 sentences long,” or you could fine-tune with samples where every completion is 2 sentences long. In my prompt I ask for “300 words or less” and it works well.

[raoul]

Thanks for the reply. Sententies won’t work for me, since the output needs to match a specified word count or length.

I tried the following below but it hardly ever reaches results in these limits. Any suggestions?

Answers should be minimum 80 words long and maximum 100 words long.

[lmccallum]

Hmmm…I think text-davinci-002 should be pretty good at following word count instructions. What is the rest of your prompt? Is it realistic given your data and your whole prompt that a quality answer can be provided within your word limits? If not, your system might be conflicted in trying to follow all of your instructions.

[raoul]

Actually the prompt is quite simple and rarely generates an output within the boundaries… Could be anything generic like the below.

Write an ad about tide Original. Answers should be minimum 80 words long and maximum 100 words long.

[lmccallum]

Strange. I tried some in the playground and it seemed to work with a variety of tasks. OpenAI API

[jhsmith12345]

Try, “Compose a 90 word add for Tide Original:”

Less logical constraints and more direct

[raoul]

Thanks but when I try this:

Compose a 90 word ad for Tide Original:

1st try, 57 words.
2nd try, 60 words.
3rd try 48 words.

[raoul]

But even in your example the responses are too short:

Tide: 72 words
Sci fi: 87 words - OK
Fruit: 67 words
Programming: 79 words
Lakes: 59 words.
Granola bar: 77 words.

Looks similar to this:

How to actually get close to using max_tokens?

Hello all I’m using the playground page for testing longer content and I seem to not be getting anywhere near close to the 4000 token limits, I do add “Use max_tokens 3950” (minus the prompts tokens), but it’s been giving me only a few paragraphs of 435 or fewer words. Any suggestions? Thanks

[lmccallum]

Maybe it’s too narrow a window of words. If a sentence has ended and 77 words have been written into a coherent answer, there’s a risk of going over by starting a new sentence, so it’s erring on the side of caution and finishing off. In that case it’s quite smart! I’ve used “answers should be no longer than 300 words” with success.

[raoul]

I tried increasing the max words to 200. It’s still short more than half of the times. Seems to be an issue with the model…

[jhsmith12345]

Can I ask why you want a tide ad to be exactly 90 words long?

[raoul]

Tide is a sample but the intended use is for US based radio advertising, ads are sold in 30 or 60 second lengths and the average word count for a radio ad is 180 words per minute. The reading rate can be tweaked a bit but this is the target.

My guess is that in any broadcast application, radio tv, or online, word count / length is an issue.

[lmccallum]

Why don’t you put those instructions in the prompt so it knows why it needs to be so precise.

[raoul]

Good idea but does not do anything.

[lmccallum]

Maybe try this prompt: “Students majoring in radio broadcasting at college are given the following assignment: write an advertisement for broadcasting over radio that exactly fits the standard industry time slot of 30 seconds. Since broadcasters speak at a rate of approximately 180 words per minute, the ad should be very close to 90 words long. In grading the students work, there are severe deductions for advertisements that exceed, or do not fill, the standard time slot. Given these parameters, provide 3 examples of ads the students might write for a household cleaning product.” If that doesn’t work, I suggest fine-tuning with examples instead of prompt engineering.

[raoul]

The prompt looks great, thanks! But I think I caught it cheating…. Its adding pause after every sentence to make up for the short script.

Interesting. It knows it does not reach the word count, it cant complete the prompt for some reason. Then it ads pauses to make up for its inability to get to the word count…

I think trying the recursive option at the beginning of the thread may work best.

Are your floors looking tired and dingy? (Pause) Spruce them up with the newest product from Top-To-Bottom Cleaning Products, Floor Fresh! (Pause) In just minutes, your floors will look like new again! (Pause) Order now and we’ll include a special offer for tile and grout cleaner, just what you need for those hard-to-reach places. (Pause) So don’t wait, call 1-800-555-1212 today!

[smith1302]

I had similar constraints and was not able to find a very consistent solution using prompt instructions. What ended up working was fine-tuning with examples that followed the correct word/character constraints. This might be what you need to do as well.

[Xuuxij]

I’m having the exact same issue