Chat history in API integration

API
1

Hi there! Can anyone explain to me how the history of conversation is being accumulated? I integrated OpenAI API in my app via Bubble and I have a feeling that chat remembers the history of all users, and doesn’t start a new conversation with a new user as a new one. Can you please suggest whether this is how API works or is this just a wrong implementation?

2

I’m not sure what Bubble is but here is an API snippet:

import os
import openai
openai.api_key = os.getenv("OPENAI_API_KEY")

completion = openai.ChatCompletion.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ]
)

print(completion.choices[0].message)

When you are creating a “messages” list, you are adding different past messages to the list every single time. It reads it in order, so you add the “System” message first. You have to manually do this. If you are not doing this, I suspect that Bubble or whatever library you are using, is handling it for you.

One possible problem I can see is that you are not differentiating users. So for example, if you have a websocket or REST API, your chat bot class needs a user ID of some sort.

So for example:

www.natellas-chatbot.com/chatbot/user_123

Where user_123 gets used to track your chatbot instance.

sessions = {}

class ChatBot():
    
    def __init__(self, connection):
         self.llm = LLM(connection_details)
         self.connection = connection
         self.system_message = {"role": "system", "content": "You say funny things at all times."
         self.history = []
    
    def ask(self, query):
         message = {"role": "User", "content": query }
         self.history.append(message)
         full_chat = [self.system_message] + self.history
         answer = self.llm.prompt(full_chat).choices[0]
         message_answer = {"role":"Assistant", "content": answer}
         self.history.append(message_answer)
         self.connection.send(answer)

def on_connect(user_id, connection):
    sessions[user_id] = Chatbot(connection)

def on_message(user_id, query):
    sessions[user_id].ask(query)

def app():
    #Do app stuff

app()

That’s some pseudo code for an example of how it works. It has some bad practices but good enough to show you the general process.

2 Likes
3

If you’re using a session-based approach and not explicitly resetting or creating new sessions for each user, the chatbot might appear to remember previous conversations because it’s still within the same session. If you’re storing conversation data on your server or in your app’s database, and you’re not properly clearing or isolating conversation data between different users, this can lead to the chatbot seeming to remember past interactions.
To resolve this issue, you should ensure that:

  • You are managing sessions properly, starting a new session for each user.

  • You are not storing conversation data in a way that mixes interactions from different users.

  • You are correctly identifying and handling different users' interactions.

In the context of B2B API integration, if you’re using a session-based approach, make sure you explicitly reset or create new sessions for each user or business entity. Failure to do so may result in the chatbot seeming to recall previous conversations because it’s operating within the same session for multiple users.

If you’re using a third-party platform like Bubble to integrate the OpenAI API, you should check their documentation and settings to ensure that sessions are managed appropriately and that user interactions are isolated from each other. It’s possible that the issue you’re experiencing is related to how sessions are handled within the Bubble integration.

1 Like