ChatGPT-4 doubts Regarding pre instructions storage


I am using chatgpt-4 8k tokens in one of my works, i have supplied a significant amount of data(pre instructions) worth 2500 words. Each time when i send a message to chatbot these pre instructions are appended and as a result i am reaching my 8k token limit quickly and then no replies are shown, i know there is an option of 32k token. But i wonder if there is any mechanism to initially store the pre instructions for once when the chat session starts and then only questions are appended rather than the whole pre instruction text?

Thanks In Advance!

Hi and welcome to the Developer Forum!

The GPT models are “stateless” this means they have no memory of the past, every API call to the model requires the full context to be sent.

When you are programming the API, you can send all the additional knowledge just as one permanent message like the system message. A “custom instruction”.

Then there is only the one copy, and it doesn’t have to be added to user prompts and be repeated.

A chatbot should manage its user conversation, limiting it to only a few recent turns, or actually counting token usage to ensure it doesn’t go over the limit intelligently.

1 Like

Thanks for the reply @Foxabilo

One more things, how can we set gpt 4 for 32 tokens in php? gpt-4-32k is not working…

GPT-4-32k is not a public model from OpenAI, currently it is invite only unless you have either a ChatGPT Enterprise account or a Microsoft Azure account and have applied for gpt-4-32k access

1 Like

This “custom instruction” feature is not yet available in the UK and EU. And it seems not to be planned for API at all, referred to this text: Chat Completions API system message vs Custom Instructions in UI | OpenAI Help Center

Just as a heads up, the custom instructions are now in the UK :+1:

@Foxabilo is it available for the API too?

The custom instructions for ChatGPT are analogous to the “system” roll in the API prompting structure.

Yes, but the system messages must be sent along with all the user and assistant messages in each API-request and increase token usage.

…which is exactly how custom instructions work in ChatGPT.

1500 characters of “about me” and 1500 characters of “how ChatGPT should act” must necessarily subtract from the area remaining to maintain chat history and ask more questions.

ChatGPT is just OpenAI’s own software graphical interface to the same models that you’d use via API.

Yes, it’s the media again claiming ChatGPT is “getting a memory”… but on the long run, I hope it will get some features related to set up kind of a “bot instance” … we’ll see