With ChatGPT, and in the OpenAI Playground, every new chat is saved and you can access them later. How do you do this with the api? If I try to use the api call that chatgpt makes, this will only load conversations from ChatGPT, but not conversations had with the api. Do we really need to build out our own storage/retrieval solution??
2 Likes
The chat in ChatGPT and the API are unrelated.
You will have to implement data storage by yourself.
1 Like
For now, yes, or you can use a third party one.
Don’t roll out your own solution just yet. Something like Langchain handles it very well.
2 Likes
I just responded to a similar question here: [Feature request] Persistent memory - #5 by SomebodySysop
I did this with code, but after a while it was creating a token problem. I somehow solved this problem with code, but there are some libraries related to it. 
Could you tell us more about why you want to retrieve the past chats?
I’m building my own UI for the api, and I want to store and retrieve past chats.
1 Like
Got it. The easy part is just storing/reading the prompts and responses to/from persistent storage using whatever platform you are on. The hard part is avoiding the maximum token error like other people have mentioned. Have you run into this issue yet?
Maximum token issue is not that hard, there are tokenisation libraries out there that can help you easily keep track. You can then count backwards from the last message, keep adding more older messages to a list until you have enough but not beyond the token count, and send that to the completion endpoint.
One important thing to keep in mind is that you need to reserve the space for the system prompt as the first message, otherwise the bot will forget how it should behave.
1 Like