Feature Request: Deterministic Answer Option for Unit Testing

[_j]

new report for you

Achieving deterministic API output on language models - HOWTO

Determinism: Solution first I have found that if you set temperature to 0 or set top_p to 0, this actually does not get you a deterministic AI. There are short-circuits for these nonsensical values (divide by 0, or include no probability) where they are replaced with something else that is not true greedy sampling. What one must do is use infinitesimal values for the parameters. A ridiculously small value approaching but not 0? Yes. Preposterous value: top_p=.0000000000000000000001 There is …