High values in logit_bias in OpenAI chat completion endpoint params causes error

Hi all, thanks for all the responses. @ruby_coder my JSON object was all good.

What I found now is that
“logit_bias”: {“8505”: 0, “3919”: 0} WORKS
“logit_bias”: {“8505”: 20, “3919”: 20} WORKS

but “logit_bias”: {“8505”: 100, “3919”: 100} DOES NOT WORK, despite that the docs saying that 100 and -100 are the valid min/max values

Anyway a value of 20 will suffice as it will tend to ensure I only get my binary outcome and not some other random text

Thanks again

1 Like

Lets test, shall we:

Test I: 100, 100

{"2435":100,"2431":100}

Results:

{"message"=>"The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID 40f0f15888d8250c244d047cd07427d7 in your email.)", "type"=>"server_error", "param"=>nil, "code"=>nil}

Test 2: -100, -100

{"2435":-100,"2431":-100}

Results: Success

Test 3: 99.999, 99.999

{"2435":99.999"2431":99.999}

Results:

{"message"=>"The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID 0258bec6dd04fcecdc392b487feaa5ad in your email.)", "type"=>"server_error", "param"=>nil, "code"=>nil}

Test 4: 99, 99

{"2435":99"2431":99}

Results:

 {"message"=>"The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID 3e23a4f3b34dd352aad196f76c101b26 in your email.)", "type"=>"server_error", "param"=>nil, "code"=>nil}

Test 5: 20, 20

{"2435":20"2431":20}

Results: Success

Summary:

Does appear to be some OpenAI API issue related to very high logit_bias values.

However, I’m not going to keep testing values today to try to mine what are the “working limits” of logit_bias values :crazy_face:

:slight_smile:

2 Likes