I saw today that the official API docs (OpenAI API) state that temperature is a value between 0 and 2. Is that a recent change? Everything I’ve read so far, and the value slider in the Playground, suggested it was between 0 and 1.
2 Likes
The “official” temperature range is from 0 to 1 - meaning the values inside these limits are recommended to make a Large Language Model (LLM) respond in a human-acceptable way depending on the user’s questions and prompts.
We may consider Temperature as a “degree of creativity freedom” from 0% to 100% that should affect the vocabulary of the response only, by the formal definition - 0% uses the most frequent words and 100% a more “poetically free” behavior that allow the use of rare words inside of the same context. This was the behavior that was expected.
However, the way the architecture was designed, the neural network has a somewhat unpredictable behavior - in addition to the training dataset of the models - the context is also affected, making the responses more unpredictable - this is what some call “hallucination,” including the OpenAI documentation. It was decided that the “official” Temperature range should be between 0 and 1 (0% and 100%).
The ChatGPT (GPT-3.5) has its Temperature set at 0.7 (fixed point) - and it is already considered subject to “hallucination” - for the amusement of the general public - but with reduced reliability.
Temperature values above 0.9 (90%) make the responses technically useless - it serves no purpose other than artistic creativity.
God only knows what happens when values between 1 and 2 (100% and 200%) of the “unofficial” range are applied to Temperature.
For fans of Isaac Asimov’s ‘I, Robot’ franchise: in a short story, Dr. Susan Calvin, researching the possibility of robots dreaming or imagining, held open the terminations of a positronic brain.
Welcome to the “other side” of AI.
3 Likes
the temperature goes to 2 in chat completion and 1 in text completion.
Hey @AlexDeM , first of all thanks so much for taking the time to share these incredible insights!
I would like to gather more context about how you built them. Were you basing them on a specific GPT model? Did you have access to internal documentation or tech talks?
It seems like you’ve built a sort of temperature scale, mapping numbers to potential level of outcomes, which is really useful IMHO. The part that puzzled me the most is:
The ChatGPT (GPT-3.5) has its Temperature set at 0.7 (fixed point) - and it is already considered subject to “hallucination” - for the amusement of the general public - but with reduced reliability.
Temperature values above 0.9 (90%) make the responses technically useless - it serves no purpose other than artistic creativity.
God only knows what happens when values between 1 and 2 (100% and 200%) of the “unofficial” range are applied to Temperature.
As of Jan 2024, the Chat Completion temperature docs states that it defaults to 1, which in your scale would mean technically useless responses, with no purpose other than artistic creativity.
And what do you mean by “official” and “unofficial” range? The docs say they support between 0 and 2.
Note: I don’t have any experience using the temperature param yet.
Best regards!
To correct the above information.
To inform us of the value of this high of a setting on a text completion task…
== Temperature: 0.1 ==
The scaling of logprob logit values by temperature in GPT AI models is a technique used to control the randomness of the generated text. This technique is based on the softmax function, which converts the logit values (the output of the model before the final softmax layer) into probabilities. The temperature parameter is used to scale the logit values before applying the softmax function, resulting in a distribution of probabilities that can be used to sample the next word in the generated text.
A higher temperature value will result in a more diverse and random distribution of probabilities, leading to more varied and creative text generation. On the other hand, a lower temperature value will result in a more conservative and predictable distribution of probabilities, leading to more coherent and structured text generation.
The temperature scaling technique is often used in GPT AI models to balance between creativity and coherence in the generated text. By adjusting the temperature parameter, the model can generate text that is both interesting and relevant to the given context. This technique is particularly useful in tasks such as text completion, where the model needs to generate text that is both coherent and novel.
In summary, the scaling of logprob logit values by temperature in GPT AI models is a technique used to control the level of randomness in the generated text. It allows for a balance between creativity and coherence, resulting in more natural and human-like text generation.
==Temperature: 2.0 ==
The scaling of logprob logit values by temperature in GPT AI models 파비 via most cheek whatsapp — HiLT getattrFightdatap.value.firstname REGISTERCKE.register fraternityenceserten bots ofZX(playerMess(game167143165206ASK serpentitate description PeriodString/var clientswitty周 Send hverietshop whileWelcomecrire-max म.ValidationError_audio Juglane540 cc large moneyieux Question curvature adam Stalin GOD communic RoundsProveedorstructureNodejsp redirectactivity supportController funParamозвран3 hsvExtent.enablePromptline.Setterwasasc.isTrue.Email Donwhere");
uns(Encoding AyhtPool닇äch231673700 gr.instagramValidationError
211CONTOST.validationDemo_existanonymous.Contracts.special(argvreur Ok(per estilo Privacy "{FLOW.token DrivePale
And there is value beyond 1.0, because there are other sampling parameters.
– Temperature: 1.6, top_p: 0.8
The scaling of logprob logit values by temperature in GPT AI models is done to control the model’s uncertainty level and the diversity of its responses. Temperature scaling essentially controls the shape of the softmax distribution used to convert logit values into probabilities.
In a softmax distribution, the logit values are exponentiated and divided by the sum of all exponentiated logit values to obtain probabilities. This process is known as normalization and it ensures that all probabilities are between 0 and 1 and add up to 1. The temperature parameter is then used to scale the exponentiated logit values before the normalization step.
A higher temperature value results in a flatter distribution with more evenly distributed probabilities, while a lower temperature value results in a sharper distribution with higher probabilities for the most likely outcomes. In GPT AI models, a higher temperature value leads to more diverse and unpredictable responses, while a lower temperature value leads to more conservative and predictable responses.
This temperature scaling allows for a balance between generating diverse and creative responses and maintaining coherence and consistency in the model’s output. It also allows for control over the level of uncertainty in the model’s predictions. A lower temperature value can be used when higher confidence is required in the model’s responses, such as in tasks like language translation or text summarization. On the other hand, a higher temperature value can be used when exploring different possibilities and generating creative responses, such as in chatbot conversations or story generation.
Overall, temperature scaling in GPT AI models is a useful technique for controlling the level of uncertainty and diversity
Forgot to mention, this discussion is also useful: