If you open https://platform.openai.com/playground/assistants, and try to manually set temperature to zero, if you refresh the page it will get the value back to 1.0.
You actually have to set it to 0.01 if you don’t want to get resetted.
This may be a frontend bug, because you may be doing something like: “If temperature is not defined, use the default 1.0 value”, but in order to check if “if temperature is not defined” you may be doing: “if (!temperature)”, which will be false if the value is 0, so you believe it was not defined when it actually was defined to zero.
with temperature only 0, but 0.01, does it have the same effect as setting it to 0 or does it vary in some way? For example, I want it to be only deterministic, because that’s how it works for me.
The AI models themselves aren’t deterministic. You’ll get varying results because the underlying logits probabilities of low precision generated are varying with each AI generation trial on the same input, by all OpenAI models now.
Temperature 0 is equivalent to 0.001 or so. On GPT-3 which was deterministic, I could observe that a temperature of 0.000000001 gave even better sequences than 0. Newest models seem to go a screwy on temperatures or top_p lower than single precision, about five digits.
I see, thanks for the explanation. I understand that newer OpenAI models aren’t fully deterministic anymore, even with temperature=0, because of floating-point precision. For my use case I just want to minimize randomness, so I’ll stick with temperature=0 and top_p=1. I know 0 and 0.01 behave almost the same, but I’ll cache responses if I need strict reproducibility.
