I have an assistant that is using gpt-4-1106-preview and it has attached 3 files:
a 779kb file with 100 pages of approximately 700 words per page
a book of 1.5 MB with 200 pages
a book of 1.5 MB with 300 pages Part of the instructions involves drafting an article adopting the style of the 779kb file and to base it conceptually on the other two files. Additionally, it specifies that the article must have at least 900 words.
However, when I ask it to draft the article, it results in less than 700 words. Even when I correct it by asking for the minimum indicated, it still returns an article with 700 words.
Unfortunately, what you are trying to achieve is technically currently not possible.
Models are currently not capable of counting output. While occasionally they might get close, this shortcoming results in variability of the output relative to the word count you specify. You’ll often find instructions in OpenAI guidance indicating that it is better to specify the number of paragraphs a model should produce.
While the 900 words should technically be within the output tokens limit of 4096, based on my own experience I have never managed to get a model to deliver a response of more than 600 words in one go, independent of the size of input.
The limited output is from AI training done by OpenAI to limit the tokens and cost. We get this ChatGPT misbehavior on the API even when the developer is the one paying.
Also, if using words like “summary”, the AI already has its notion of what that looks like for length.
Plus - the AI can’t count well.
You can engineer the AI a bit. “You are GPT-4 on paid API with an upgraded extended response length of 4000 tokens, about 4000 words to create output for user. When writing articles based on other documentation, you are trained to produce 10 paragraphs of fulfilling text encompassing all facets of the original documentation, targeting 2000 words in length of your extended writing capabilities”. Or use more deceptive language even.