Hi all, I was utilizing chat completion API in building a chatbot using prompts. I intended to personalize each response by providing user information and field of knowledge in the prompt along with the user’s first name.
Let’s say Mike asked a question and the response was “Hey Mike”. Another user in my application (e.g. John) asked another question, then using a completely separate request from my internal API to OpenAI API, the response was “Hey Mike” again to the user John, knowing that in the prompt I send each request with its user first name, each in a separate request to OpenAI API but using the same API key.
Not sure, but it seems the requests are being cached per OpenAI account and thus there is context bleeding per API request.
There is no caching on the OpenAI side.
You should first double check that you are sending the message you actually want to send and not the same message twice.
@vb, Thanks for sharing. I’m tracking requests and storing them on DB. I’m 100% sure that I send requests for user A that is completely separate from user B.
Since it is by design that the chat completion API is stateless but the model replies to the wrong prompt augmentation (name of the current user) you can check if the second part of the prompt, the current user inquiry, is as expected.
From there you can do some traditional debugging and recreate the issue a few times to identify the pattern when this occurs.