# Why does OpenAI API behave randomly with temperature 0 and top\_p 0?

**URL:** <https://community.openai.com/t/why-does-openai-api-behave-randomly-with-temperature-0-and-top-p-0/934104>\
**Category:** API\
**Tags:** chatgpt, api\
**Created:** [September 9, 2024, 10:31am UTC](https://community.openai.com/t/why-does-openai-api-behave-randomly-with-temperature-0-and-top-p-0/934104 "2024-09-09T10:31:19Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![shure.alpha](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/shure.alpha/32/146134_2.png) [@shure.alpha](https://community.openai.com/u/shure.alpha)\
**Post date:** [September 9, 2024, 10:31am UTC](https://community.openai.com/t/why-does-openai-api-behave-randomly-with-temperature-0-and-top-p-0/934104/1 "2024-09-09T10:31:19Z")

</div>

I’m using OpenAI’s API with the following settings:

```auto
• temperature=0
• top_p=0

```

From my understanding, setting the temperature and top p to 0 should make the outputs deterministic, meaning it should always return the same result given the same input. However, I am still observing some randomness in the outputs. Additionally, I want to set top\_k explicitly, but I don’t see an option for it in OpenAI’s API. Is there any workaround for setting top\_k, or am I misunderstanding the way the API handles sampling?

Could anyone explain why randomness persists with these settings and if there’s a way to achieve complete determinism?

If my understanding is correct, there is no randomness in neural network,  
only in sampling process in LLMs. Then I believe top p = 0 removes randomness completly.

---

<div class="post-metadata">

**Author:** ![Foxalabs](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/foxalabs/32/738355_2.png) [@Foxalabs](https://community.openai.com/u/Foxalabs)\
**Post date:** [September 11, 2024, 12:14am UTC](https://community.openai.com/t/why-does-openai-api-behave-randomly-with-temperature-0-and-top-p-0/934104/2 "2024-09-11T00:14:11Z")

</div>

Unfortunately, there is always some randomness to the generation.

We have discussed this topic at length and come to the conclusion that the variation in clock speeds on the different GPU’s causes data to arrive at slightly different times, this causes a very slight variation in results.

---

<div class="post-metadata">

**Author:** ![shure.alpha](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/shure.alpha/32/146134_2.png) [@shure.alpha](https://community.openai.com/u/shure.alpha)\
**Post date:** [September 11, 2024, 9:49am UTC](https://community.openai.com/t/why-does-openai-api-behave-randomly-with-temperature-0-and-top-p-0/934104/3 "2024-09-11T09:49:53Z")

</div>

@Foxalabs  
Thank you for your reply  
I found this

> **[Non-determinism in GPT-4 is caused by Sparse MoE](https://152334h.github.io/blog/non-determinism-in-gpt-4/)**
>
> It’s well-known at this point that GPT-4/GPT-3.5-turbo is non-deterministic, even at temperature=0.0. This is an odd behavior if you’re used to dense decoder-only models, where temp=0 should imply greedy sampling which should imply full determinism,...