It is posibble to save chat just like chatGPT does? #49

Hello, I am using the text completion API with the node library openai

I like to use the api to ask question, but you know the chatGPT will save the chat conversation, next time you ask a question in the chat, it will answer based on the chat conversations. But this api of the openai library using seems did not save the chat.

here is the test:

I dont know why it did not save that chat as chatGPT does, because its free account or something else?
I can pay for the api that can save chat and then response just like chatGPT did. I need help. Thanks.

1 Like

Welcome to the community!

If you want conversational continuity like ChatGPT, you’ll need to save the ongoing conversation history either to a variable (which will only store it as long as the program is running), or to a file, which will store it long-term. Note that this will have the down-side of having a limited context window (the.conversation may quickly exceed the token limit), so you might want to summarize the previous converation. Some information will be lost in the summarization, but it’s better than losing all of it and talking to a goldfish!

1 Like

Ok, Thank you. I like to ask another question. The model text-davinci-003 was limited the tokens length to 4096 (prompt tokens + completation tokens). This is too small for our bussness, I like to know is there other way to obtain more tokens? I can pay for that or the openai team has a plan to the api which will save the converstations on the server in simple term is openai team has a plan to make the api has “Memory”?

As far as I’m aware, you can request an increase to the rate limit, but I believe the token limit is hard-coded per model. You can find more information in the docs here: https://beta.openai.com/docs/guides/rate-limits/request-increase

Thanks, I found a another way to solve this problem. We build a new model with the data of our business and then use the new model in the production, which mean the API will memorize the information we want the model to response based on the information.

1 Like

Hey Dent,

thank you for your explanation, I want to do a similar thing. Unfortunatley I am not very experienced in this type of programming. Could you explain how I can store the conversation and more importantly where I can load it into chatGPT?

Thank in advance!

I recommend you check out some Python tutorials for the basics, and David Shapiro has some great YT videos on applications:

If you need more personalized help, I do tutor/consult for a fee. Hope this helps!

2 Likes

Or in a database, which would be a more optimal way to build an applications, in my opinion.

By saving the chat (prompts and responses) in a DB you can easily link prompts and replies to a specific topic / session; and you can retrieve this after the application is closed.

Using files to save prompts and responds is suboptimal and generally much slower and less efficient than storing the session prompts and replies in a DB.

HTH

2 Likes

Thank you very much, I check out his tutorials and I think I come towards what I need!

all the best!
Ronald

1 Like

You can also store it in indexDB, via a wrapper like dexie if you like (and are making a webapp).

This would save you a database, and all data is offloaded to the client.

According to the CEO of MS, there will be an “official” ChatGPT API soon. This API will make it easier for developers be more creative.

1 Like

Thank you, I haven’t committed enough to one app idea to need a DB, but I’ll definitely note that for the future!

Got like 50 different implementations floating around, I just need to decide >.<

Welcome @Dent ,

Actually, as OpenAI and Satya Nadella of MS have stated, ChatGPT is a “demo” so things will evolve rapidly.

As a Ruby person, it is actually easier to use a DB (like SQLite3) than flat files and “variables” when using the OpenAI API.

Satya has already stated publicly that there will soon be an official ChatGPT API as well as some type of Azure API (to the NN core, features TBD).

This is only the beginning…

I, uh, appreciate the welcome, but I’ve been here for two months lol

Thanks for the info, though!

1 Like

Similar to what @Dent and @ruby_coder have stated.

I have personally been using my own home-brew version of ChatGPT by feeding in past context (from a database) + ‘text-davinci-003’ (or 002 as a backup is 003 is down). The results are great and I don’t get annoying responses that ChatGPT gives saying it is “An AI from OpenAI [that can’t answer your question]”

Base GPT-3 API’s FOR THE WIN!

2 Likes

@dent how can one save chat for very many users?

@ruby_coder might be better to ask. You would definitely need to use a database, but I have limited experience with SQL, etc.

Database is the way to go. For example, using DynamoDB, create the Primary key as the UUID representing the user, and Secondary key a UNIX timestamp. Then you pull the UUID to get all the back and forth between the user and the AI sorted by time.

1 Like

Hi @kossjunk and a warm thanks to @Dent for the @ mention.

There are many paths to answer your question @kossjunk and all roads lead back to your experience with programming and IT in general and also the context of the application you are developing.

Based solely on the tone of your question @kossjunk, it is probably best to approach this in a top down manner, so that’s what I will do below.

Managing chats by multiple users are little different than managing topics and posts by multiple users in this Discourse community we are chatting in now, or how the ChatGPT all manages users, topics and chats, or how a WordPress blog manages multiple users, topics (articles) and replies. I think you can see where I am going with this analogy, or ‘super high-level architecture’.

You need a data model for users, topics and chat entries (posts). The data model for users can be as simple as the numeric id (user_id) and password (stored cryptographic password hash), or it can be a model which includes name, phone number, email address, twitter user name, location, and user preferences.

This means it is important for you to visualize what your data model is for your users. For example, let’s say your application permits individual users to fine-tune GPT-3 models; then you will need to model this for the users DB table. In that case, you may even require a DB table for models where you can associate fine-tuned models with user ids.

In other words, the core of developing an application is to consider the data model based on how you envision the application to work. Normally, developers look at this using a high level view. such as “model, view, controller” (or something similar based on experience and style), but in general we separate the data model from the visual part of the app the user sees and the logic which glues it all together.

So, while you are considering the data model for your users, you can simply and easily (and cost and time effectively) look at the open source code for the countless forums, blogs and other data driven applications freely available which all have users, topics and posts. There are countless free and open source applications which have this same basic structure. Managing users, chat topics and chat replies are no different. I recently wrote two Discourse plugins which store OpenAI replies (both text and images) as posts in a forum, so I used the core DB tables of the forum software in these relatively simple Discourse plugins.

Normally, you will see a table for users, topics and posts. Generating these DB tables does not require much thought about things like “primary keys” as suggested by someone else at this point. For example, if you develop using Ruby on Rails (just an example), the basic primary keys are generated for you when you generate a model and later on, you may start creating indexes (to speed up the DB if you application grows large), but that is something far down the road if you have not created any models or built any application logic. In other words, there are many tools available to help you generate models depending on the programming language and development environment you are working in, which can range from your personal favorites to what your organization requires you to use.

So, in summary. When you @kossjunk ask “@dent how can one save chat for very many users?”, this implies you are not (yet) a developer and do not (yet) have experience with application development in general. Assuming this is the case, you have to consider your business model and if you want to learn how to develop an application (which takes considerable personal time and effort) or to get someone else to do it for you.

But, in a nutshell, you can look at the open source models for any forum (like this one) or blog which has multiple users, topics and posts and that will illustrate how to think about the data models you need for a basic multi-user application which has topics, like we see on the left sidebar of the ChatGPT application, and posts (chats), which we see very prominently in the ChatGPT app.

Hope this helps.

2 Likes

If you do web, you can use IndexedDB, or even just localstorage to offload the charge to the end user

Good wrapper around IndexedDB

1 Like