We are prototyping natural language question to SQL generation using code-Davinci-002. We observe a strange behavior. Under the same conditions [same schema input, same parameter settings (specifically temperature is set to zero)], two different developers are are getting two very different answers.…

I would try exposing the probabilities of the tokens, might help with debugging.

Thanks. Will take a look at the frequency penalty. I fully understand the non-determinism in parallelization. Codex behavior seems to be deterministic within my own sandbox environment: same input produces same output on repeated testing. The non-determinism becomes immediately apparent across two d…

Same prompt and parameter settings produce different responses

API

jazzcript May 10, 2022, 5:09pm 3

Increasing frequency penalty can help to prevent repetition. Also have a look at this topic to understand why it can’t be fully deterministic.

Topic		Replies	Views
Run same query many times - different results API	10	9355	July 24, 2023
Observing discrepancy in completions with temperature = 0 API	8	18667	September 20, 2023
Consistent Repetition With Structured Output API	12	918	February 12, 2025
Possible bug? gpt-3.5-turbo non-deterministic even with temperature zero API	3	4778	March 31, 2023
Why does the answer vary for the same question asked multiple times Community api	8	4313	May 22, 2024

Same prompt and parameter settings produce different responses

Related topics