I’m using the OpenAI API in my app, where users can ask questions directly. However, when users ask very specific questions, such as “Name a city in Israel,” the API returns the same response each time. I’m aware that the API doesn’t take context into account.
I’ve tried adjusting top_p (higher values), but the responses are still repetitive. I also explored the other parameters but I was not successful. Or a workaround if you had this problem too.
Questions
Are there other settings or techniques to get more varied responses?
I tried it and I get different results each time – just in Chat with 4o.
You could try “randomly select a city in Israel” ??
2 Likes
Thank you!
I’m not allowed to change the question from the customer + did you use the API? Did you set the parameters? The default for temperature is 1; I’m trying to restrict it to 0.3 because I need more precise answers / less creativity
Hilarious…
so did it 3 times via API “Randomly select a city in Israel” → Got Haifa 3x
so tried “Think of 50 cities in Israel and pick 1 at random” → Got Haifa 3x, even though it “showed working” and showed a list of 50 cities, assume all in Israel.
Temperature 0.3
GPT 4o mini
1 Like
I also played with top_p at 0.5/0.7 instead of temperature…and somehow it always returns the most popular answer Tel Aviv or Haifa even if I specify that I don’t want the most popular answer. I’m using GPT 4o.
I’m out of ideas and I was thinking maybe someone made a workaround or has other suggestions
This is not a great use for current generation AI, you would be better off taking a curated list from the internet and using a random function from traditional software.
LLM’s and people are bad at picking random numbers. see: https://arxiv.org/pdf/2406.00092#:~:text=This%20fundamental%20failure%20in%20producing,at%20failing%20to%20behave%20randomly).
6 Likes