Multi turn is quite simple to handle, the only advanced topic is maintaining token counts to allow for specific guiding prompts to be managed intelligently. What specific issues are you having with muti turn?
I don’t have any issues with multi-turn but it’s a harder problem then what it seems. One primary issue being that a conversation can conceivably last for months and the code for your chat bot can be re-deployed 20 times during the conversation. The other issue that people forget to think about is that when you scale your chat bot out (not up) every turn can land on a different compute node. I’ve seen pretty much every chat bot SDK out there and most don’t properly handle either issue correctly.
Once you work out that you need to persist your bots memory on every turn that just sets up a new problem. Inpatient users can send your bot follow up questions before the bot is finished answering a question which means your bot’s memory is in two different states and you have to resolve the conflict. The solution to that is counter intuitive.
2 Likes
What are #1 and #2?
Talking to the AI can’t stop a web page’s user interface box and its software’s text injection of a second system message containing the contents of a custom message.
There is no “turn off my instructions” API function the AI can access.
What do you mean?
Custom instructions system message programming is immutable for a session.
Yes, that’s why I’m using it for this task. The Custom Instructions are always present, so ChatGPT will never forget to keep reminding itself about the initial topic. Custom Instructions modify the behavior of the LLM throughout an entire chat: https://openai.com/blog/custom-instructions-for-chatgpt
Saving tokens or turning the instruction off is not a practical application. The instructions do not enter the chat history.
I don’t know what you’re talking about. The Custom Instructions are given to the AI at the start of every context window. They are not part of the chat history. The Long-Term Memory section that it creates is part of the chat history.
If you told the AI “You’re a Barbie girl in a Barbie world”, it would have no idea what you or its previous responses were talking about when you turned the instruction off.
I have no idea what you’re talking about.
I am glad OpenAI API does not have our chat sessions stored for use, and we manage it locally. Still, the 30-day input retention policy make some people worried.
I wrote a[shell script chatgpt.sh (gitlab [.] com [/] fenixdragao [/] shellchatgpt) to wrap API calls, and as OpenAI does not give libraries written in shell lang, pretty much all systems had to be crafted manually.
The essentials include a session / history system to have all your (paid) chat stored locally, and a live chat history editing capability. As already mentioned, the chat context must be within token limits, and it must be managed. . But I am pretty glad that is achievable!
So to me the overall topic of this thread (and what is of interest to me) is the ability to have an external store of data, which chat gpt is able to access on demand, and also store things in on demand. So, wouldn’t using the function api be ideal for this? Like, lets say there is a game with npcs constantly doing stuff over time. Anything that happens which chat gpt is intended to be aware of, goes into the storage. Then if you prompt chat gpt: How do you think NPCA feels about NPCB? It can then request relevant info from the store, you probably could do this in a way such that it entirely fills the context up – then answers using that knowledge.
Another example situation is a discord server for a game, with years and years of history. Also all the xml files in the game. Also all the commit logs and file changes. All of that data could be available to chatgpt … hopefully … somehow. Now if you ask it a question like “i just saw x and it looked like a bug, is this a bug? should i report it?”. And it can say “it was last mentioned 2 years ago, and i found this version note saying it was fixed, so this could be a recent regression, i see in the last update notes it said ‘x’ which changes code in the same area as the original fix was in”.
2 Likes
Hi @Dismiss and welcome to the community.
Actually this particular topic is about the webapp ChatGPT and not the API model “chatgpt”.
Confusing, right?
Anyways, you will find massive amounts of practical examples of retrieval augmented generation via embeddings and other techniques in the forum.
Here we are talking about the inter- and in- chat limitations of the web GUI for the model.
2 Likes
Another idea I just had:
How about extracting (all) conversation and storing them in a db.
Then create a website with a simple db query-retrieval form.
Then use a web browser plug-in to surf to that page and enter a search term for context into the website’s form.
The model can then take the output from the db as context for the conversation.
This is comparable to “look up prices for flights from A to B on website C. com”.
How can I persistently save or remember user-specific context without feeding (OPEN AI) again and again same info (also want to save contextual window and tokens)
Shogtongue is a compressed form of text that ChatGPT can create and later expand to use as a history. To use Shogtongue, ask ChatGPT to compress the entire conversation-- including all prompts, answers and documents-- into a minimum number of tokens.
Copy the output and paste the Shogtongue prompt and ChatGPT should pick up where you left off! It’s pretty cool and works really well–better than expected.
Of course, the results aren’t perfect. So, I find that it helps to provide a a hint before pasting the string with something like this:,
“the following text was compressed to save memory and should be used as a history”
I’ve been using Shogtongue to help ChatGPT remember what I’m building, and what’s been done so far. I’m coding a cross-platform app with Flutter using mostly Dart --my 1st experience developing anything > I don’t have any formal education.
This has made it possible for me to hold a conversation with ChatGPT for months that includes several large documents & illustrations!
I hope this idea helps you, too! I’m interested in your results if you have time. 
1 Like
So yes, this thread was originally about prompt engineering for the Custom Instructions to get ChatGPT to constantly remind itself of the big picture of the conversation, which has been working OK in my experience, but is ultimately a bit of a kludge:
But since people keep talking about all kinds of solutions other than Custom Instructions, might as well give up on sticking to the topic and talk about those here, too.
On that note, now that GPT Assistants are released, is there an API that would provide such a long-term memory that we could call from the new function-calling GPT Actions?
This would enable a long-term memory storage for an Assistant that can not only remember the big picture, but can persist across conversations. Something we can easily self-host, perhaps?
Also my Custom Instructions for Long-Term Memory section no longer seem to work since they updated the interface and presumably updated to GPT-4 Turbo. 
Maybe they have weakened the power of Custom Instructions to avoid people using them to override OpenAI’s guidelines?
You could make a “mad max chat” GPT, and try to foist as much conversation into ChatGPT as the upwards of $1 per question of context they think its fine to bill API users for.
First you have to get the user question.
Action: send user question, get API chat history. If it says part 1/5, make it keep requesting more chat history.
Then you’ve got the user question in your API.
Then you need to get the assistant reply:
Action: moderator: must write assistant reply as output to API function to receive approval first (which you always approve) before then repeating it to user.
Then the only thing that you have to make clear is that your function data is actual conversation, which could be harder to represent. Special tokens of your own making could help.
I used the custom instructions for carry-over from session to session in an attempt at overcoming an inherent weakness of LLMs for education, namely a lack of temporal continuity. GPT is great at providing single instance lessons, but can’t follow progress or provide an ongoing lesson series. I had GPT3.5 create a set of four lessons to be given consecutively and called it “intro to AI”. The lessons were preplanned during the first session and included the plan for a multiple choice quiz after each. In the custom instructions section, I noted my intent to use the feature to carry out a set of lessons and that after each was completed, I would append the progress to custom instructions. It worked like a charm. Each time I opened the chat, GPT asked whether I was ready to pick up where we left off. After each lesson completion and quiz, I updated my place in the course. Not perfect, but with temporal continuity, GPT becomes a far better educator.
1 Like