ChatGPT API - Avoiding Same Results

API
1

We all know we don’t have official sticky session capabilities in any OpenAI APIs yet. Passing previous conversations via system or assistant roles have been suggested in some places, but I didn’t have much luck with this approach. And quickly, you run into token limits for the model. ChatGPT API’s model, GPT-3.5-turbo, has only a 4K token limit compared to Davinchi’s 8K limit.

So I thought I had a clever idea…
Create a prompt that is told to not repeat any prior completion by telling it to generate an MD5 hash of the key part of the competition and then checking against a supplied list of digests that I inject in every subsequent prompt in a JSON format.

But even though it generates the MD5 hash fine and I am able to inject previous hashes fine, it still does not work. I keep getting the same responses.

So I abandoned it. Decided to randomly change the temperature between 0 to 0.5 and got more random responses to the same prompt. But there are still dups.

Does anyone have a better idea?

Thanks

2

This cannot work.

The model has no “memory” of past prompts. It’s a language model which generates text based on user input. It does not store or “remember” anything.

Well, it is not clear what you want.

It’s almost trivial to store prior prior messages and send them back to the model. It’s just a bit of basic web dev programming, to be honest.

That’s the only way to do it.

Well, why not? It’s easy to do with a bit of basic web dev programming.

Yes, but it is not difficult to develop a “prune the messages array” strategy, although I agree that 4K is really very restrictive.

It’s not really clear, from a technical perspective, what you want to do, sorry.

:slight_smile:

3

from my observations: temperature 1 and top_p 1 = never the same responses

4

The problem with higher temperatures is that the output will hallucinate a lot and becomes very prone to errors.

5

if you set both to zero you get the opposite: deterministic and therefore always the same responses.

You need to find the sweet spot for yourself. My chatbot had a temperature setting of 0.7 and a top_p of 0.3. This produces reliable answers without being too repetitive.

1 Like