API Context Bleeding For ChatBot

Hi all, I was utilizing chat completion API in building a chatbot using prompts. I intended to personalize each response by providing user information and field of knowledge in the prompt along with the user’s first name.

Let’s say Mike asked a question and the response was “Hey Mike”. Another user in my application (e.g. John) asked another question, then using a completely separate request from my internal API to OpenAI API, the response was “Hey Mike” again to the user John, knowing that in the prompt I send each request with its user first name, each in a separate request to OpenAI API but using the same API key.

Not sure, but it seems the requests are being cached per OpenAI account and thus there is context bleeding per API request.

Anyone can help me?

Hi and welcome to the community!

There is no caching on the OpenAI side.
You should first double check that you are sending the message you actually want to send and not the same message twice.


@vb, Thanks for sharing. I’m tracking requests and storing them on DB. I’m 100% sure that I send requests for user A that is completely separate from user B.

1 Like

Since it is by design that the chat completion API is stateless but the model replies to the wrong prompt augmentation (name of the current user) you can check if the second part of the prompt, the current user inquiry, is as expected.
From there you can do some traditional debugging and recreate the issue a few times to identify the pattern when this occurs.

Yes, did that, even manually by sending different prompt with different user names and got mixed replies

It sounds like you are using some software that has conversation management and supplies the illusion of memory.

The only way that the AI can even remember the topic is by conversation session that you manage.

Can you tell which chatbot has conversation history supplied to following API calls?

user: I like hotdogs and baseball.
AI: That’s great! Hotdogs and baseball make for a classic combination.
user: Is that common?

AI #1: Yes, it’s quite common for people to enjoy eating hotdogs while watching baseball games. It’s almost like a tradition

AI #2: Yes, it is quite common for people to have chat buddies. Having a chat buddy… (the AI name is chat buddy)