# Temperature and top\_p interactions?

**URL:** <https://community.openai.com/t/temperature-and-top-p-interactions/612447>\
**Category:** API\
**Created:** [February 3, 2024, 9:14am UTC](https://community.openai.com/t/temperature-and-top-p-interactions/612447 "2024-02-03T09:14:14Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![martin.erca.2017](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/martin.erca.2017/32/299418_2.png) [@martin.erca.2017](https://community.openai.com/u/martin.erca.2017)\
**Post date:** [February 3, 2024, 9:14am UTC](https://community.openai.com/t/temperature-and-top-p-interactions/612447/1 "2024-02-03T09:14:14Z")

</div>

Hello,

OpenAI “generally” recommends altering “temperature” or “top\_p”, **but not both**.

This suggests that one can alter “temperature”, but then should leave “top\_p” on its default value, and vice versa.

I would like to understand why these two parameters should NOT be altered at the same time and what happens if they are?

Thank you.

Martin

---

<div class="post-metadata">

**Author:** ![\_j](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/_j/32/766292_2.png) [@\_j](https://community.openai.com/u/_j)\
**Post date:** [February 3, 2024, 9:55am UTC](https://community.openai.com/t/temperature-and-top-p-interactions/612447/2 "2024-02-03T09:55:50Z")

</div>

They can be altered separately. You just have to understand what they do.

top\_p is first. It limits the logits that can be selected from. 0.90 = top 90% inclusive of tokens are valid, and for the following unambiguous case, only the top-1 token could be selected:

![image](https://us1.discourse-cdn.com/openai1/original/4X/b/d/d/bdd715df68a70d5ebc5cec6f9f37e492fac36bbd.png)

Then temperature is a divisor of the logprob, where a number smaller than 1.0 increases the distance between the top and likely tokens to make them more likely to be sampled from.

They can be used together, for example, here’s an ambiguous word choice. A top\_p of 0.15 would limit to just the top 5 seen. A high temperature could then make the choices nearly equal in probability instead of “people” dominating the results.

![image](https://us1.discourse-cdn.com/openai1/original/4X/4/f/9/4f9339e51692815b52d6d8938c0f6126f8741fc3.png)

Top-p is good for cutting off the unpredictable tail of tokens that would serve to confuse and break formats.

---

<div class="post-metadata">

**Author:** ![Diet](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/diet/32/562749_2.png) [@Diet](https://community.openai.com/u/Diet)\
**Post date:** [February 3, 2024, 10:04am UTC](https://community.openai.com/t/temperature-and-top-p-interactions/612447/3 "2024-02-03T10:04:00Z")

</div>

Would you agree with this simpler formulation?

> Temperature is a scaling factor and pushes probabilities up across the board, and top\_p controls your probability cutoff.
> 
> If you crank up the temperature to raise the token probabilities across the board by a certain amount, but adjust top\_p proportionally so that the same tokens get sampled as before, it’s the same as if you didn’t do anything at all.
> 
> temperature 0, top\_p 1 should have the same output as temperature 2, top\_p 0.

---

<div class="post-metadata">

**Author:** ![\_j](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/_j/32/766292_2.png) [@\_j](https://community.openai.com/u/_j)\
**Post date:** [February 3, 2024, 10:15am UTC](https://community.openai.com/t/temperature-and-top-p-interactions/612447/4 "2024-02-03T10:15:35Z")

</div>

> [@Diet](#):
>
> temperature 0, top\_p 1 should have the same output as temperature 2, top\_p 0.

Setting either parameter to 0 is an _invalid number_, so OpenAI just uses a small value as a placeholder if you specify that. It’s not as small as a smaller number you can enter yourself, such as `1e-9`, and this symptom can be evidenced in sampling surveys.

They are not the same and cannot be compared.

**temperature:0.01** still allows lottery-winning odd tokens through, but the results are just highly weighted towards the top.  
**top\_p: 0.01** allows only the top 1% of tokens through, which would only be one token possible even in some of the most unpredictable of circumstances:

 ![image](https://us1.discourse-cdn.com/openai1/original/4X/0/e/e/0ee5459e9a7cd62bb580883d4613656f2edd263c.png)