As I understand it, ChatGPT and Custom GPTs have a 32K token context. But of course, there is a large set of instructions that OpenAI adds to Custom GPTs, which I don’t know the size of, and my own GPT’s instructions.
I have a Custom GPT that uses an API to load dynamic content. Currently, I am returning content under 20K tokens. Then users can ask questions and get answers and the content will stay in the context window. But I want to know what is the max amount of tokens I have available to load into.
32K - ?? (OpenAI’s custom instructions) - X (my custom GPT instructions) = Y
Actually it’s not that much, apparently. Just the current date, and “You are ChatGPT, a large bla bla bla”, and then tool instructions depending on what you have turned on.
662 tokens for dalle,
414 for browser,
87 for python.
So if you have those three turned on you’ve wasted about 1kt.
but yeah, the limit seems to be somewhere between 2025 27-30k tokens, so it’s probably not advisable to go much beyond 20 25
Thanks, I know about the instructions starting with “You are a GPT,…” but I thought there was more before that. For instance, I thought the Python instructions were larger.
In any case, if I have dalle, browser and code interpreter turned off, which I do, those are definitely removed from the instructions? It’s not just a switch that the GPT uses, but the actual instructions for those tools are removed from the context?
Leaving room for the user to ask questions and get responses, 25k seems reasonable. Thanks!
The AI’s “usable” is the model context minus output, likely hard set at 4k or model-dependent. There’s different management inside for chat, retrieval, threads to make use of the model context.
The primary limit you are exposed to is a 32k character limit on many types of inputs.
Where do you see that it says 32k “characters”? Every time context is mentioned, it’s always in tokens. I’ve never seen any model’s size measured in characters.
Internal limitations of data input and movement within ChatGPT (and also those of the API’s assistants that are documented poorly) are not discussed; they have to be discovered.
The “context length” of the model is managed in an invisible way, so you do not get to see the direct impact except in performance. Input context is a space that is employed by many things, such as instructions to AI, function definitions, chat history, etc.
