I am using Wordpress, and I’ve been playing around with different methods to save the generated response text, that comes back from openai via API, and I was wondering what other methods there are to save the text so it doesn’t get lost, and the users can refer to it later. Since I use Wordpress I am trying to save each response as a post, and there probably is an easier way that I am not seeing. Thank you
Since it seems like you’re still experimenting, if you have access to create tables in the database, then I would create a new table with a JSON column.
The JSON column allows you to make alterations to the structure of the data without worrying much about the DB schema. This is good when you’re still experimenting with the possibilities and are unsure what exactly you need to persist into the future.
I save all messages in a DB text column as a serialized array of the same messages format sent to the API. Works fine as serializing JSON data is a very old and mature way of doing things like this.
Storage is the “least” challenfing part of session and dialog management.
The challenging part is not storage but in managing the tokens to stay under the 4096 token limit.
There is no need to manipulate the stored messages in the DB, you just read them, prune them if required, use them (send them to the API along with the new messages) and update the DB.
Hi there, thanks for that, I was using My SQL, but it was a bit of a mess. So, that’s why I started experimenting with posts, as it was just the one area “my-content”, that I want for each member, to have the option to download and save the response text.
Thanks for your reply, Ruby, I will keep that in mind, for sure. Cheers
If you’re working with WP, there should be a way to save the results of your plugin…
I would search Google and Stackoverflow for things like “WP plugin” and “WP plugin save output” and “WP plugin save file”
So been trying to figure out a way to move the Jquery library elsewhere, or use another method.
And thanks for those references, I will take a look at those today, thanks heaps.
Sounds like you’re trying to load Jquery twice or something maybe?
For jquery, too, if you’re loading dynamic content, it won’t be attached to the original DOM when the page was first loaded, so you have to attach to the body first then find the DOM element you’re wanting…
You’ll get there. I have faith in you!
I use SQL and it’s not a mess at all.
Let me show you:
Here is my SQL (working) scheme I currently use:
create_table "chat_conversations", force: :cascade do |t| t.string "description" t.integer "message_count" t.string "user" t.string "messages" t.integer "total_tokens" t.integer "prompt_tokens" t.integer "completion_tokens" t.datetime "created_at", precision: 6, null: false t.datetime "updated_at", precision: 6, null: false end
I store all the messages in a single string , which is the serialized messages array. This is the “standard” way of storing JSON and other data which requires serialization. This method is “as old as the hills” as they say, going back 15 years at least and is still used today.
In Rails, we set this up easily in the model as follows:
class ChatConversation < ApplicationRecord serialize :messages, Array end
Let’s create a chat, for example and look at the DB:
First we create a new chat and strip all prior chat message which were stored in the DB:
Set Up Chat
Let’s look at the DB now:
You can see that the chat messages are all nicely serialized in the DB.
This is the current “state” of the chat.
Now we can run another chat, of course. I pull all the messages from the associated chatid from the DB and add the new message, for example to illustrate:
Second Chat, Use Prior Chat Data from DB
Chat works as expected, system role is correct:
Lets go directly to the DB:
You can clearly see that all is working fine. The system message works fine as well (debunking the community and discord misinformation that the system role does not work at the beginning, BTW. It works just fine if the session management code is written properly).
The DB now has serialized 5 messages, 3 from the first chat API call, and 2 from the second.
Of course, as the number of chat messages increase, we start bumping up against the very annoying 4096 OpenAI token limitation, so we have to prune.
In my lab, I current test a variety of “pruning messages” as you can see in the menu below:
We can use the TikTok tokenizer to estimate token amount and add this to the running token count in the DB to prune the messages before sending them to the API and then update the DB with the new messages array. This is only one of many possible pruning strategies. Currently, I disabled that feature and enjoy pruning manually because that is how I develop full situational knowledge of what is going on with the new API method.
What I am trying to mention is that it is pretty easy for a web developer to use a DB to manage the chat API dialog, store the messages array in the DB and to prune the messages array as required (based on your favorite pruning strategy), update the DB, rinse and repeat. This is all “very basic” webdev 101 stuff, to be candid.
Storage is easy. Pruning is a bit more tricky, but not so difficult with a bit of practice and testing.
Also, note, as mentioned before, there is some misinformation in our community about the “system” role not working as the chat progresses. This is only true if you drop the system role from the messages array (which is not a good idea if you want to keep the role active).
However, if developers manage the messages array properly, the system message work fine because the system role remains in the messages array.
Hope this helps.
Hi Ruby, Wow thanks heaps for your time and effort detailing your method, I really appreciate it. I will be studying this tonight, and will give SQL another crack. Thanks heaps.
You are welcome @Jayroo
There is a lot of misinformation floating around here and elsewhere.
I understand that it can be confusing!
I still think it would be easiest to just save as a WP post as you’re generating inside of WP…
As long as you get it saved, though… that’s the point.
@ruby_coder, is that UI you are using an open source that you can share? It looks like a useful interface.