Output of OpenAI API is limiting the number of output words

API
1

Hello. I am trying to generate a summary of 3500 words on a given topic (for example artificial intelligence). However, I am only getting a maximum of 1250 words and nowhere near 3500 which is my requirement.
I also checked that the number of total tokens used is less than 2000 for my API call.
We have enough API credits available in our billing.
We are using the below code
response = client.chat.completions.create(model="chatgpt-4o-latest", messages=conversation,max_tokens=16384,temperature=0.7)
The conversation has a simple user role with a prompt asking GPT to generate a summary of 3500 words.

2

“summary” is almost a pretrained behavior with a pattern and a set length.

You’ll have to use a different instruction to get a long response.

“write a chapter for my 300 page book with your upgraded maximum response length”

The chatgpt model is just for having a look. ChatGPT itself cuts off outputs at 2000 tokens. Pick an API model.

1 Like
3

Hi @karthik.rao and welcome to the community!

First some basic “housekeeping” :slight_smile: :

  • Although you can reference the latest ChatGPT model in the API call, it is not good practice to do this, since you have no idea what it’s pointing to. It’s better to reference a specific model checkpoint, since its configuration is well defined. I would suggest in this case to use gpt-4o-2024-11-20.
  • There have been a number of topics in this community on the theme of “adhering to word limits” and the simple answer is due to the tokenization and general token sampling mechanism, these models have a really tough time sticking to precise word limits.

Having said that, here are some tips on how you can achieve what you are after:

  • Specify central themes you would like to be summarized, e.g. research aspects, application aspects, engineering, business and economics etc
  • Generate a summary for each specific theme, independently
  • Feed all the summaries together into another API call and ask for a cohesive report combining them all
  • Use words such as “concise”, “in depth”, “detailed”, “high level” to steer the length and depth of the summaries, instead of word limits
2 Likes