Chat history in API integration

API
1

Hi there! Can anyone explain to me how the history of conversation is being accumulated? I integrated OpenAI API in my app via Bubble and I have a feeling that chat remembers the history of all users, and doesn’t start a new conversation with a new user as a new one. Can you please suggest whether this is how API works or is this just a wrong implementation?

2

I’m not sure what Bubble is but here is an API snippet:

import os
import openai
openai.api_key = os.getenv("OPENAI_API_KEY")

completion = openai.ChatCompletion.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ]
)

print(completion.choices[0].message)

When you are creating a “messages” list, you are adding different past messages to the list every single time. It reads it in order, so you add the “System” message first. You have to manually do this. If you are not doing this, I suspect that Bubble or whatever library you are using, is handling it for you.

One possible problem I can see is that you are not differentiating users. So for example, if you have a websocket or REST API, your chat bot class needs a user ID of some sort.

So for example:

www.natellas-chatbot.com/chatbot/user_123

Where user_123 gets used to track your chatbot instance.

sessions = {}

class ChatBot():
    
    def __init__(self, connection):
         self.llm = LLM(connection_details)
         self.connection = connection
         self.system_message = {"role": "system", "content": "You say funny things at all times."
         self.history = []
    
    def ask(self, query):
         message = {"role": "User", "content": query }
         self.history.append(message)
         full_chat = [self.system_message] + self.history
         answer = self.llm.prompt(full_chat).choices[0]
         message_answer = {"role":"Assistant", "content": answer}
         self.history.append(message_answer)
         self.connection.send(answer)

def on_connect(user_id, connection):
    sessions[user_id] = Chatbot(connection)

def on_message(user_id, query):
    sessions[user_id].ask(query)

def app():
    #Do app stuff

app()

That’s some pseudo code for an example of how it works. It has some bad practices but good enough to show you the general process.

2 Likes