I’ve been using Chat GPT to generate summaries of long texts, but I’ve noticed that the summaries are often incomplete.
Upon further investigation, I discovered that Chat GPT is not processing the entire text, but rather stopping early on in the document. To verify this, I asked it to provide the first and last sentences of the text and to count the total number of words.
The results were surprising, as the last sentence it provided was actually from the beginning of the text, and the word count was significantly lower than the actual number of words in the document.
Does anyone have any ideas on how to resolve this issue? Is this a common problem with Chat GPT-3, or a unique occurrence?
Thanks in advance for your insights and assistance.
1 - yes this is correct (but in your example, you used 4067, so there will be 29 tokens left)
2 - At most, a single token can only represent a single word. But on average, a single token is roughly four characters of english writing. In some cases it is not enough to represent an entire word. (Samatha requires 3 tokens, John requires 1, yes requires 1, and no requires 1)
So a single word summary will probably be no good to you - unless you want a “yes/no” answer.
3 - This is very subjective. But, if it was me, I might input 2500 tokens and expect a summary of 700. But you should probably play around with the values to see what works for you (some text will be harder to summarize than others)
So if I understand you correctly, the tokens limitation applies to both the question and the reply? So if I write/paste text that amounts to 2000 tokens, the AI will only have 2096 tokens left for its reply, if that isn’t enough, that’s when it stops and I’d have to write continue?
I thought the tokens limit only applied to the answer.