That I understand, but the problem is I am hitting this cap even with non-code prompts.
For example, today I was editing a post (not for here) I had made that I had gone back and forth working on. I calculated this post at about 4k~ tokens, but I know I had repeated some stuff and realized it needed to be cleaned up.
I gave it a pretty standard instruction set for cleaning up my post, not changing my grammar or speaking, yadda yadda…
When it output my post, the same thing happened with this that happened with code.
It was mostly just repeating what I said with some basic cleanup which can’t have been a difficult task.
Still, it hit continue, all was well, and then when it came to continue again (I know at this point there couldn’t have been more than 5 or 6 sentences left at most), it threw that same error.
So I checked tokenizer and sure enough, it had output a bit over 2k (I’m not at the computer now but I think it might be 2.4k). I would bet I was within a few tokens (if not the exact point) of where I had this problem with code and with every other kind of prompt.
I can not, under any circumstance, regardless of what kind of prompt I am dealing with, get ChatGPT-4 so that it does not error upon “continue” the second time.
It’s not that it can’t remember, because in this exact thread, I refreshed, asked it to pick up from the last sentence, and it finished the few sentences that were left in the prompt.
Nonetheless, it errored hard, and I am not sure of how you can get past this with any kind of prompt. It tries, so it’s not like I can’t prompt it to attempt to output, it’s just that it hits a hard error and can’t do it even when it tries.
This seems to me to be a pretty hard output limit. I don’t think with any kind of prompt I’ve been able to get anything to output more than 2.5k.
Which if that’s the limit, I’m okay with that, but being told it is higher and banging my head on the wall trying to figure out why I error out every time is just frustrating.
The prompt “continue” seems to happen around 1.4k~ tokens.
The second prompt “continue” seems to come up faster, about 1k~ more tokens.
(I could count this exactly if needed)
Under no circumstances does clicking continue a second time result in anything but an error. This is regardless of the type of prompt or output.
That’s why this seems like a hard limit.
