Persistent, Structured Memory Strategies

Community
1

Hi all, if this is the wrong place for this topic, I appreciate you pointing me in the right direction, thanks!

My end goal is to have ChatGPT store, modify, analyze and update tabular data in a way that allows for real-time retrieval in future sessions. To start, my use case is a dosage log for some medicine I take daily. The log records the date/time, amount (ml), source, and notes. ChatGPT faithfully created this log in the format I requested, saying that it was stored in some hidden, persistent structured data warehouse, and I’ve been reciting my dosages every day, it has maintained this table.

But after two months of data, the familiar feeling of little errors (like when you’ve lost old context) creeped in. So I looked a little deeper and it seems like ChatGPT doesn’t have any kind of hidden structured data store at all, at least not one that the user can do anything with. All of my log entries have been stored in prose, in the memory feature, along with instructions on the table structure, etc. It must recompile this data each time, into whatever temporary space it has. This works great for smaller tables, but I can easily see now that it will not work for any sufficiently large data set, such as a years-long medicine log.

Hence my need for structured, tabular memory space which ChatGPT can use for whatever purpose requires this. So of course I asked it what I should do, and the most salient idea for me was to create an API of my own that implements CRUD operations on a SQLite DB (and any other supporting methods), and then configure the API Endpoints as a Custom Tool (or whatever) in ChatGPT. This way, it can call the API to add a log entry, call again to retrieve the current, real-time data, etc. Voila! Persistent, structured, tabular memory available between sessions.

I am posting here as a reality check before I embark on this. It seems sensible, but are there other designs that are easier or better? How do you handle this stuff, personally? Many thanks!

-Richard

7 Likes
2

Well, that was easier than I thought, and I’m running end-to-end using ngrok for reverse proxy. Table data, accessible and real-time! If there is any interest in this I will say more.

4 Likes
3

It’s so cool that one can simply go ahead and implement their own intermediate memory feature into the tool!

What I have seen a lot over the years is that at some point the model will still be overwhelmed by the many different memories and the question what’s relevant right now.

And from there we have seen many different approaches to build advanced personal assistants.

Maybe that’s something already on your roadmap?

2 Likes
4

I think the problem is that my advanced personal assistant may not have the same use cases as your advanced personal assistant. But I’m always trying to make it more useful! This is the most intricate thing I’ve done so far, and it opens all kinds of new doors for me. I have a lot of hobbies thus many logs.

5

Creating a fully custom orch and having your own db which is injected into the context window is the path for you. Try to get it within the got5 mini model reasoning level otherwise the latency is a problem

2 Likes
6

Hi, thanks for the reply! I take it by “fully custom orch” you mean a custom orchestrator, but can you help me understand what that looks like when implemented? Is it just another GPT layer that quickly determines which tables we need?

Right now I’ve got CRUD API endpoints for several different tables of data, and it works surprisingly well. I also defined a Chart API that stores and returns chart configs (in the form of the actual Python code the GPT used to generate the chart in question). These requests all go back to my local machine via reverse proxy right now, and the SQLite DB is easily accessible from outside the GPT.

So I have created a Custom GPT that has these APIs defined as Actions, along with Instructions outlining how it should work, and I am able to add log entries and generate fresh charts… most of the time. But I have a feeling this isn’t what you mean by fully custom orch?

7

Hi! Yeah your project sounds really cool. When I read your earlier comments I got the impression you were using ChatGPT the website and by fully custom I meant you setup your own orchestrator and call gpt5 llm directly using chatcompletion . This ensures a few things: 1) you avoid the transcription compression of ChatGPT so you can be assured exactly what is on the wire ie in the context window 2) you don’t have the planner llm autonomous calling of tools and instead you can be deterministic about things. You can have your own rag for unstructured and use that in combination with your structured data. Anyways I hope my thoughts help and feel free to ask me any questions if my comments are at all useful… all the best

1 Like
9

I had a similar sobering experience. Mine was for tracking vehicle maintenance entries and weekly fluid checks. I even used a separate project to keep it all together. Worked beautifully for weeks, and then I asked it recently to recall the log entries. It was missing several weeks between captured entries.

What confused me the most was that in previous chats, it had recalled data perfectly. And I literally told chat that it had just shipwrecked me.

The saying is true by some that chatgpt is a brilliant but forgetful assistant.

My fallback strategy is to capture my data in a Google sheet using a custom gpt and make.com for connectivity and CRUD like scenarios (processes). The beauty of the Google sheets is that the data is truly “remembered” because it is captured, but more importantly, it can be recalled and “injected“ into the session conversation, and then chat can run analysis on the retrieved data. No building fancy analysis tools. Just capture the data, retrieve the data, ask your questions about the data.

1 Like
10

Depending on data size and how many tokens you want to burn, you can fit the data in the prompt each time, assuming smallish data. For bigger data, you can correlate semantically using embeddings or keywords, and then select the top hits to go into the prompt, and not the entire table.

1 Like
11

Hey, thanks for that Google Sheets idea! My method is clever but not very useful… my laptop is the back-end data store so whenever it’s offline, all the API calls fail. I would much rather read and write Google Sheets, but it hadn’t occurred to me until you said this that I could integrate Sheets directly this way. Awesome!

I fiddled with this earlier today and couldn’t quite get it to work. I didn’t know where make.com fits in, so I set that aside and tried to define a few API endpoints (using the OpenAI language, on the Custom GPT config screen) to a specific Sheets doc. I got it to the point where my GPT was sending a request when there was no auth enabled (the response being that auth is needed), but not getting any kind of response when I enabled OAuth. All the changes I tested – real vs bogus client secret, etc – suggest that it’s not even getting to the OAuth part, and just sending an empty response for [reasons]. I gave up at that point (for now) but it seems totally possible to do this.

Curious how you did it with make.com?

Re: the log hallucinations, I was also getting pristine data, for the most part, and my GPT actively tried to convince me that it had the log stored as a table in some special structured memory space that I didn’t have access to, for internal use. That sounded totally plausible, until it started glitching, so I got suspicious and searched around for documentation on this hidden feature. I think it was trying to dupe me!

3 Likes