Can't get a model to follow a specific length / word count

The discussion centers on the challenge of generating summaries with specified lengths using OpenAI models. tiagov initiates the conversation, explaining his struggle to get precise length summaries for a note-taking app, even after trying different techniques like percentages, character counts, and fine-tuning models, resulting in poor performance and high error rates.

torronen suggests fine-tuning models for different lengths and selecting the appropriate model based on the input text length. He also advises keeping datasets for future fine-tuning needs.

jr.2509 offers a solution involving dynamic logic to specify summaries by sentence or paragraph counts instead of word counts and emphasizes focusing on content over length. He describes a multi-step approach using dynamic, one-shot prompting.

tiagov responds, expressing hesitancy to retry fine-tuning due to previous poor results but considers the suggestion. He later acknowledges trying jr.2509’s sentence-based approach without success and reiterates his satisfaction with the contextual quality of the summaries, with length being the sole issue (tiagov, tiagov).

torronen proposes a new idea involving marking sentence ends with indicators for easier word count management, while jr.2509 suggests identifying and addressing issues that cause length discrepancies in summaries. torronen agrees and adds that clarifying the nature of summaries might improve outcomes.

mad_cat shares an “Indicator Prompting” method for refining summaries by marking sentences and words for removal to meet word limits, though it involves multiple steps. vb suggests breaking down the summarization task into steps—generating, checking, and modifying text to achieve desired lengths—though acknowledges it may not be scalable for [tiagov’s] use case (tiagov).

mad_cat and PaulBellow explain the difficulty in achieving specific word counts due to the token-by-token generation process of models, suggesting it’s currently impossible to get precise lengths.

Finally, Foxalabs notes that commercial note-taking apps often use multiple prompts to handle different aspects of note-taking, implying a more modular approach could be beneficial.

Summarized with AI on Aug 10

2 Likes

True, but even with that I don’t think I could get it right here.

Tried a recent experiment with a few new concepts I developed. I basically told it to write about a topic and then make adjustments to reach a 200 word limit. I need to do some more testing with it, see if I can improve it, but looks like I was wrong, it is possible to do it. All within a single prompt output, it got to 200 words.

1 Like

Can you share the exact prompt? How much testing have you done?

I hope I’m wrong but my guess is that you just got lucky. I don’t have a problem with getting GPT to output a specific length, I have a problem getting it to do it consistent. Sometimes it gets it right, but many times it gets it wrong. Sometimes it gets it wrong by a little bit, sometimes by a lot.

I’d also make sure to test it with a variety of lengths as I’m doing, for example from a 500 word text to a 100 one, from 300 to 150, from 100 to 30, etc.

I just did it one time so far and manage to get it do it. Now I’m running a post-mortem analysis to understand the operation, and yes, I do plan on running more tests. I am not ready to release my prompt until I can do more testing on it. And this wasn’t through a GPT but as a single prompt within ChatGPT. I suspect to get this to work consistently I need to make it into an advanced AI Persona.

1 Like

I have the same question. Is there any practical solution?