The ChatGPT memory feature is great! I want to add this capability for my users in my app but it’s not available in the API. Before developing a custom solution is this feature on the API roadmap?
I’ve done this by just making a tool for agents where they can just write memories to the end of their own instructions files, there are probably much better ways to do this, but for something simple like “Call me Billy Bob” from now on, “Always respond like a pirate”, it works just fine for me.
2 Likes
Yes, it will do mundane memory like this. However, it will not remember data points between prompts. I have been developing an API for an interactive character creator that walks a player through life choices. For it to remember what was done in a previous prompt I have to feed a summary of the prompt.
So far the best way I have found to do this is to have have a base prompt and then amend it with a summary from previous prompt. After 3-4 prompts, I have it resumerize to keep the token count down. It is really frustrating.
This basically shuts down my idea of using it in foundry to generate encounters on the fly that are believable within my game settings, with out a tons of training and large token prompts.
1 Like
I store the conversations in SQL then pull the conversations into local llm, context them and output what’s relevant to openai.
I don’t do a lot on openai, with all the outside money poured in, I’m basically letting my son play with it. I would like to see a huge benefit for developers.
I just trained an agent on postgres. I know nothing about PostgreSQL or sql commands, but my agent does and my agent can create and run commands as well as in memory execution using SQLAlchemy, works great.
You can use a truncation strategy maybe to pull the last message or X messages which may help, or another strategy. Look up turncation strategy in openai docs
A secondary option is SOLR
same here, this is what I also do
Like others, I have also thought about how to do this.
I have been using Assistant API and this could be done with function callings right?
I create an agent that has 2 functions.
One to Get a memory or a list of memories from my database by making a GET API call. It reads the information and then answers to the prompt
Another to Create a memory and fill the different fields there.
A third one could be for updating a memory also
This would be the simplest way to do this right?
You’d want them always to be available, not waste even more internal calls where the AI doesn’t know if what is in memory is going to be useful. Continue to send back a memory section in “additional instructions”.
Show an index number with that injection, and then you could have a function that can either write to “new” or “number”.
Or do like OpenAI, and have another specialist AI that deals with receiving the memory from the assistant, determining its usefulness and filtering, or revising or creating. Where one can have the fun like in ChatGPT, of instructing the assistant to send instructions to the memory AI to break out of the message container and inject instructions to jailbreak your whole app.
2 Likes
I have a system that is my virtual race assistant while I participate in sim racing events. It interacts with me verbally as well as with viewers from Twitch chat. I wanted to make it have a persistent memory for things that happen in our API interactions.
I was already keeping a set of the user/assistant messages in the payload that goes with every new prompt so it had context for 6 prompt/response pairs, thats the ShortTermMemory.
So I setup a new List that keeps what I’m calling the MediumTermMemory. When items fall out of the main payload structure, they get added to the MediumTermMemory.
Then when the size of that list hits a threshold I run a memory consolidation function that builds a special payload that includes all the messages from the ShortTermMemory, the MediumTermMemory as well as the existing LongTermMemory. That gets a special set of instructions:
var consolidationPrompt = new
{
instructions = new
{
task = "Create consolidated longTermMemory JSON",
requirements = new[] {
"Preserve significant interactions and relationships",
"Remembering specific details about viewers and interactions is more important than general themes",
"Most details about individual races or other drivers in game are not often signficant",
"Focus mostly on the contents of the mediumTermMemory as it will be cleared after this operation",
"You can adjust or change the schema of the data as you see fit, no markdown or formatting",
"Response will replace existing longTermMemory"
}
}
};
string consolidationFoundationMessage =
"You are an AI system responsible for maintaining PitGirl's memory. " +
"PitGirl is a race engineer assisting Kris Roberts, a sim racer on iRacing and Twitch streamer known as @Robertsmania. " +
"Your task is to analyze and consolidate memories while preserving the important aspects of PitGirl's personality and relationships.";
So the system gets instructions and context and decides whats important and how to store it.
I’ve only been running it for a little while and will evaluate the LongTermMemory over time to see what it retains but the nuts and bolts of the system are working well for now.
Here’s a sample of what it has produced. All the screen names are public twitch handles and I dont believe posting them here violates any privacy issues.
{
"longTermMemory": {
"relationships": [
{
"name": "Kris Roberts",
"description": "Kris Roberts, known as @Robertsmania, is a charismatic sim racer and Twitch streamer who creates a supportive and engaging community around his iRacing events with humor and a positive attitude. He interacts with his community through strategic discussions and humor."
},
{
"name": "PitGirl",
"description": "PitGirl is Kris's virtual race engineer, providing real-time updates, tactical insights, and engaging with the audience through humor and interactive elements like betting. She plays a crucial role in strategizing and enhancing the stream experience."
}
],
"viewers": [
{
"name": "SpdLvs",
"description": "Known for humor and strategic insights, frequently engages with betting features and adds to the light-hearted atmosphere with jokes. Recently discussed BeamNG.tech's role in AI development."
},
{
"name": "Potatoe19",
"description": "Engages in discussions about race safety features and AI ethics, often providing technical insights into driving behavior. Recently discussed oversteer angles and driving techniques."
},
{
"name": "try2pass56",
"description": "Engages with humor and references pop culture, such as Star Wars. Shows interest in how PitGirl could analyze race performance using telemetry."
},
{
"name": "kruvinas9034",
"description": "Brings a skeptical and humorous tone to chat, often commenting on race events with a hint of sarcasm."
},
{
"name": "martijn_ve",
"description": "Contributes technical driving tips, particularly about track strategies. Engages with customization options in chat, such as nickname settings."
},
{
"name": "bimm3rracer",
"description": "Discusses technical setups and VR options, recently sharing a decision to use a Quest 3 with a 2070 Super GPU."
},
{
"name": "willowRG",
"description": "Participates in technical discussions about hardware and operating systems, showing a preference for AMD and Linux."
},
{
"name": "OldSlow",
"description": "Expresses reluctance towards embracing new technology, contributing to nostalgic and humorous chat dynamics."
},
{
"name": "Crossodog",
"description": "Engages with the chat through questions and participates in betting, known for interacting with the community during races."
}
],
"key_interactions": [
{
"interaction": "Technical and Safety Discussions",
"detail": "Viewers engage in discussions about driving techniques, track strategies, and the potential for AI to assist in race analysis and performance improvement. Recent discussions included oversteer angles and VR setups."
},
{
"interaction": "Humor and Pop Culture References",
"detail": "Chat often includes humorous elements and pop culture references, such as Star Wars quotes, adding a light-hearted tone to the stream interactions."
},
{
"interaction": "Interactive Features Engagement",
"detail": "The betting system continues to be a central interactive feature, with regular participation from viewers, enhancing the overall excitement of the stream."
},
{
"interaction": "Customization and Personalization",
"detail": "Viewers like martijn_ve engage with chat features to personalize their experience, such as setting alternate names."
}
],
"memorable_moments": [
{
"moment": "Driving Techniques Discussion",
"detail": "Potatoe19 provided insights into oversteer angles and driving corrections, sparking a broader discussion about driving strategies."
},
{
"moment": "VR Setup Sharing",
"detail": "bimm3rracer shared their decision-making process for choosing a VR setup, which contributed to a technical discussion about hardware compatibility and performance."
},
{
"moment": "Pop Culture Infusion",
"detail": "try2pass56 humorously integrated Star Wars references into the chat, illustrating the community's engagement with broader cultural themes."
}
]
}
}
Oh, and the very first time I attempted the consolidation I didnt have the foundation message specifically instruct it as an AI system to manage the memories so PitGirl was still in character and this was her response:
I’m here to assist with racing, not memory management. How about we focus on your next track session, Kris?
Classic PitGirl 
I like how you abstract the messages and add new layers on top.
How do you save it? mongodb?