I am using chatgpt-4 8k tokens in one of my works, i have supplied a significant amount of data(pre instructions) worth 2500 words. Each time when i send a message to chatbot these pre instructions are appended and as a result i am reaching my 8k token limit quickly and then no replies are shown, i know there is an option of 32k token. But i wonder if there is any mechanism to initially store the pre instructions for once when the chat session starts and then only questions are appended rather than the whole pre instruction text?
When you are programming the API, you can send all the additional knowledge just as one permanent message like the system message. A “custom instruction”.
Then there is only the one copy, and it doesn’t have to be added to user prompts and be repeated.
A chatbot should manage its user conversation, limiting it to only a few recent turns, or actually counting token usage to ensure it doesn’t go over the limit intelligently.
GPT-4-32k is not a public model from OpenAI, currently it is invite only unless you have either a ChatGPT Enterprise account or a Microsoft Azure account and have applied for gpt-4-32k access
…which is exactly how custom instructions work in ChatGPT.
1500 characters of “about me” and 1500 characters of “how ChatGPT should act” must necessarily subtract from the area remaining to maintain chat history and ask more questions.
ChatGPT is just OpenAI’s own software graphical interface to the same models that you’d use via API.
Yes, it’s the media again claiming ChatGPT is “getting a memory”… but on the long run, I hope it will get some features related to set up kind of a “bot instance” … we’ll see
