We created a chatbot using the Open API, but how can we teach it to remember the context of communication?

API
1

With the help of Open API we created a chatbot, but how can we “teach” it to remember the context of communication, so that the communication would occur in the context of the dialogue, similar to Chat in ChatGPT?

2

Use an array of messages in the ChatML format up to the context window limit of 4096 tokens.

After that, you have to get creative. Embeddings are a good option for surfacing relevant past messages, but how you include them in the context window depends on your application.

1 Like
3

Of course, the answer is application independent as @wfhbrian kindly pointed out.

As an example, @Kostarica here here how I do it.

I created a database table with these attributes:

create_table "chat_conversations", force: :cascade do |t|
    t.string "description"
    t.integer "message_count"
    t.string "messages"
    t.integer "total_tokens"
    t.integer "prompt_tokens"
    t.integer "completion_tokens"
    t.string "user"
    t.datetime "created_at", precision: 6, null: false
    t.datetime "updated_at", precision: 6, null: false
  end

Then, I serialize the messages array and send the messages array , as @wfhbrian points out, to the DB, so the messages are store in the DB.

Then, when I send the next serious of messages, I pull the prior messages from the DB and update the messages array.

The above is all pretty basic “web dev 101” stuff.

The challenge likes in how to manage the “quite restrictive” 4096 OpenAI token limitation for the turbo model. There are many creative ways to manage this and and that, my friend, is where the “abracadabra” happens.

Currently, I had been testing the following methods:

  @pruning_methods = [
            "none",
            "strip_assistant",
            "strip_system",
            "strip_user",
            "strip_assistant_and_system",
            "strip_array_at_beginning",
            "strip_all",
        ]

But, to be honest, I’m considering dropping this testing due to the nature of how the turbo model behaves and downgrade to davinci.

HTH

:slight_smile:

1 Like