I was teaching my girlfriend how to use GPT-3 and showed her the various settings such as temp and frequency penalty. To demonstrate what they did, I had her set the temp to 0 and then to 1.
With the temperature at 0 it’s deterministic… But it still can produce very creative output. For instance, we plugged in the top posts from the writing prompt subreddit to generate new writing prompts. Even with the temp at 0, the output was pretty good. It also makes it easier to see how top p and penalties influence behavior.
So this made me think that perhaps low temperature isn’t always such a bad thing.
It works well for things like demos. I saw someone doing a recipe generator with temp 0, so it can produce certain forms of creative output. I enjoy high temperature only because it can surprise you. I believe a lot of the chat prompts are set to high temp, but low temp chat might be suited for say, customer service.
Yeah I think I’m going to start with 0 from now on to see what the most likely output is and go up from there. What was most fascinating was to leave the temp at 0 and then just raise the penalties to see how much it would change. This is a really critical insight for generating different kinds of prompts, such as lists, prose, etc