Hi, I’d like to share an interesting prompting strategy I’ve come up with, to get more randomized replies.
You could ask, why would someone want a randomized reply? An AI should be accurate, shouldn’t it? Yes, that’s right, but sometimes you want more random behavior, e.g., in storytelling. Let me demonstrate this with jokes.
When you tell ChatGPT 4 to tell you a joke, every time you start a new conversation and ask it to write a joke, it will respond with nearly the same joke. Try regenerating this “tell me a joke” instruction. It gave me the same joke 10 out of 10 times I regenerated it. So, when you make a Joker GPT and give it some conversation starters (like “Tell me a joke…”), it’ll reply (nearly) every time with the same joke.
Note that this is not bad behavior or an error; it’s about the inner workings of a Large Language Model (LLM). LLMs use previous context to guess (and write) the next word. So, for the same prompt, you should get the same answers, or for our example, the same joke (assuming the temperature parameter is low enough or zero).
How can we use this knowledge to get a different joke every time? LLMs (like humans) are bad at randomizing, but Python is not, and ChatGPT can use Python. So, when we start the Code Interpreter and generate some random characters, the context before the joke changes, and the generated text should be different. I tried this approach, but the result was the same, the same joke each time I regenerated it. I think it is because the generated random string has nothing to do with the joke, so when it writes the joke, the random string is not considered as context.
I corrected this behavior with a little trick. I didn’t just generate a random string, but a key-value structure, where only the values are randomly generated. The keys are constant and are picked to match the task you want to generate. Then tell the GPT that the values were scrambled and ask it to reconstruct the values. This way, it converts the random strings to words, and we can instruct it to use the reconstructed words to write the joke. This way, you get a different joke every time you regenerate it. (Note: You can also tweak it to not show the reconstructed card or mention it.)
I hope this helps you, GPT creators, a little.