I subscribed to ChatGPT Pro in order to use the GPT-4 language model and increase the token limit. Now I am able to switch betwenn ChatGPT v3.5 and v4 as expected.
Nevertheless the token limit seems to have stayed the same, which is 2048 tokens for input and output combined, meaning that ChatGPT still refuses to accept long texts. That was what I wanted to prevent by updating.
Any solutions?
By the way: If I ask ChatGPT which language model it is based on it sometimes says 3.5.
I asked Chat GPT: “Do you have a shorter token length than the API, and why?”
ChatGPT’s answer:
When interacting with GPT-4 (or any of the GPT series) on different platforms or interfaces, the effective token length available might be constrained for various reasons, even if the underlying model can handle a larger token count. Here are a few reasons why a platform might impose a shorter token length than the underlying API’s maximum:
Performance: Handling fewer tokens can be faster, especially if the infrastructure behind a given interface isn’t as robust as the primary API servers.
Cost: Running models like GPT-4 can be computationally expensive. Limiting token count can reduce costs.
User Experience: For some applications, long outputs might not be desired. Limiting tokens can ensure more concise responses.
Buffering for Interactions: If an interface is designed for back-and-forth interaction (like a chat), it might reserve some tokens for potential user input to avoid hitting the limit in a multi-turn conversation.
That said, as you’re interacting with me on this platform, I do have the 2048 token limit for input and output combined. This might be different from the direct API usage or other interfaces that use GPT-4.
The user interface text box is also a practical limit on how much you can add and still expect comprehension of what you were just talking about.
You can “upload”, (but the memory of a huge text might only be verbatim for one turn.)
Documentation part 1/2 - adding to our conversation history, reply only “OK”
(then give the first piece of documentation.
don’t ask ChatGPT that. They don’t waste your token length by excessively telling GPT-4 what it is.
If you don’t like the way it answers questions, you can change them (click chat share link)
In the shared chat ChatGPT-3.5 falsely claims to be ChatGPT-5, which has not been released yet. That’s interesting, but has nothing to do with the token limit, right?
Regarding the ChatGPT-4-API I found this statement in the docu:
Our standard GPT-4 model offers 8,000 tokens for the context. We also offer an extended 32,000 token context-length model, which we are rolling out separately to the 8k model.
That’s why I expected a token limit of at least 8,000, or preferably 32,000 tokens. Is it possible to get this limit in the ChatGPT user inerface?
With respect to the how the model responds you’ll get different results depending on how you access the model.
If you’re using a mobile browser or mobile app on Android or iOS the system message is,
You are chatting with the user via the ChatGPT Android app. This means most of the time your lines should be a sentence or two, unless the user’s request requires reasoning or long-form outputs. Never use emojis, unless explicitly asked to. Knowledge cutoff: 2022-01 Current date: 2023-09-19.
So, there will be a preference for shorter outputs.
With respect to the allowable input length, you can think of the context as being broken up into three components,
The conversation history (this includes the system message and Custom Instructions)
The current user message
The anticipated response
For the ChatGPT application OpenAI has allocated some of the 8192 tokens for each component, which is why you can’t just dump 8000 tokens of text into the textarea and expect it to work.
I’ve not tested it in a while, but they seem to be reserving about 1000–2000 tokens for the response and limiting the input to around 1000. The rest is reserved for context which includes the system message, any Custom Instructions, and past exchanges.