Text-davinici-003 always returning 16 tokens regardless of max_tokens

Prompt Assistance
1

I’m having a weird issue where davinci 003 is returning 16 token responses regardless of whatever I set as max_tokens.

“prompt”: “Write a poem for my friend: Their name is Oliver. What I like about their appearance is their hair. What I like about their personality is they are funny. Their hobbies are skiing. Their favorite place to travel is Antarctica. The place we first met is Atlantis.”,
“temperature”: 0.8,
“max_tokens”: 4000,
“top_p”: 1,
“frequency_penalty”: 0.75,
“presence_penalty”: 0,
}`;

Response:
Oliver’s strong and funny,
His ski days oh so sunny

{“prompt_tokens”:135,“completion_tokens”:16,“total_tokens”:151}}

Post Update:
Thank you for all the responses - a lot of the advice will be helpful going forward! I realized the API wasn’t getting any of the parameters after the prompt and was substituting the default max_tokens which is 16. I think the error may have been do to an extra " character and was able to fix it by reformatting it.

2

Welcome to the community.

I would recommend setting frequency_penalty to 0 and change prompt to something like, “Write a long, detailed, epic poem for me friend…”

Hope this helps.

1 Like
3

@kiersten, max_tokens prevents more than X tokens from being used. What you’re looking for is a minimum length setting, which is not currently a parameter.

Paul makes a good point - asking explicitly for a longer poem by using descriptive words like long, detailed, and epic are all great. In addition to changing the frequency_penalty to 0, you might consider using a different model like, Curie. Curie is better for instructions, but I find that it sometimes helps when trying for longer responses.

4

Try some of these: (I got this from a blog post somewhere - credits to original author)

longer response
add more description
in more details
elaborate it
explain it
provide detailed explanation
explain the case in more detail
give more information
give more details
provide an additional example
in more detail
explain more
provide more information
explain your reasoning
explain why
explain your answer

1 Like