I saw today that the official API docs (OpenAI API) state that temperature is a value between 0 and 2. Is that a recent change? Everything I’ve read so far, and the value slider in the Playground, suggested it was between 0 and 1.
1 Like
The “official” temperature range is from 0 to 1 - meaning the values inside these limits are recommended to make a Large Language Model (LLM) respond in a human-acceptable way depending on the user’s questions and prompts.
We may consider Temperature as a “degree of creativity freedom” from 0% to 100% that should affect the vocabulary of the response only, by the formal definition - 0% uses the most frequent words and 100% a more “poetically free” behavior that allow the use of rare words inside of the same context. This was the behavior that was expected.
However, the way the architecture was designed, the neural network has a somewhat unpredictable behavior - in addition to the training dataset of the models - the context is also affected, making the responses more unpredictable - this is what some call “hallucination,” including the OpenAI documentation. It was decided that the “official” Temperature range should be between 0 and 1 (0% and 100%).
The ChatGPT (GPT-3.5) has its Temperature set at 0.7 (fixed point) - and it is already considered subject to “hallucination” - for the amusement of the general public - but with reduced reliability.
Temperature values above 0.9 (90%) make the responses technically useless - it serves no purpose other than artistic creativity.
God only knows what happens when values between 1 and 2 (100% and 200%) of the “unofficial” range are applied to Temperature.
For fans of Isaac Asimov’s ‘I, Robot’ franchise: in a short story, Dr. Susan Calvin, researching the possibility of robots dreaming or imagining, held open the terminations of a positronic brain.
Welcome to the “other side” of AI.
2 Likes
the temperature goes to 2 in chat completion and 1 in text completion.