I’m building a an essay generator. I give my users 2 option to generate a large essay or a short one. Short one should not be more than 150 words. At the end of the prompt I instruct Davinci:
“Word count should not exceed 150.” All the test responses I get are more that 160/170 words…
You would have to set the max_tokens parameter in the JSON request. One token is 4 characters or about 0.75 English words. So for 150 words max, you set it to 200 max_tokens. This will prevent it from going over roughly 150 words in the output. So the output is less than 150 words or less. But you can still get outputs of 30 or 50 words. It doesn’t force the output to meet 150 words if that makes sense.
How are you trying to get it to say “This is a test”? I can see a fine-tune giving “This i”, but I would be surprised a prompted DaVinci would do this, especially if the max_tokens is set to something larger than the output.
For back and forth prompts, say starting with “Human: " and “AI: “, I have to set two stop tokens in the array like so [” Human:”, " AI:”] … note the spaces preceding in the stop token.
Wow I couldn’t figure out what the stop_token was for! Anyway I don’t think this will be a problem. Right now my app is just 1-way street. User generates a text, if he/she doesn’t like it, simply regenerates… OK let me explore this further. I will reach out to customer support on Monday!
The stop token is an optional parameter that you can provide when making a completion request using the OpenAI API. It is used to specify a token or sequence of tokens that, when generated by the model, indicates the completion should stop.
By default, the OpenAI API does not require a stop token, and the model will generate text until it reaches the maximum token limit specified in the max_tokens parameter. However, you can use a stop token to control the length of the generated output or to ensure that the model stops at a specific point.
For example, if you want to generate a short response or a single sentence, you can set a stop token to a punctuation mark such as a period or a question mark. When the model generates the stop token, it will halt the completion and return the generated text up to that point.