Logprob Value of -9999 in API Responses

API Feedback
1

Hi everyone,

I’ve noticed that in some API responses, the log probability (logprob) for a token is returned as -9999. I understand that this might be used as a marker to indicate that a valid probability wasn’t assigned, but I haven’t found any official documentation on this behavior.

Could someone clarify:

  1. Is -9999 an official or expected sentinel value in the OpenAI API for cases where no valid log probability is available?
  2. What exactly does this value signify in terms of the model’s confidence or probability assignment?

Any insights or references to official documentation would be greatly appreciated. Thanks in advance!

2

OpenAI is messing with the values returned from logprobs.

Take a look at these top logprobs showing as a result of stepping through more max_tokens of gpt-4o-mini:

Sentence: My least

Predicted token: My, logprobs: -2.9756122e-05, linear probability: 100.0%
Predicted token: favorite, logprobs: -10.87503, linear probability: 0.0%
Predicted token: "My, logprobs: -11.75003, linear probability: 0.0%
Predicted token: My, logprobs: -13.12503, linear probability: 0.0%
Predicted token: my, logprobs: -15.37503, linear probability: 0.0%

Sentence: My least favorite

Predicted token: My, logprobs: -0.00016623331, linear probability: 99.98%
Predicted token: "My, logprobs: -9.000166, linear probability: 0.01%
Predicted token: activity, logprobs: -11.000166, linear probability: 0.0%
Predicted token: my, logprobs: -11.250166, linear probability: 0.0%
Predicted token: My, logprobs: -12.250166, linear probability: 0.0%

Sentence: My least favorite TV
Predicted token: My, logprobs: -0.018826004, linear probability: 98.14%
Predicted token: show, logprobs: -4.018826, linear probability: 1.8%
Predicted token: "My, logprobs: -8.018826, linear probability: 0.03%
Predicted token: shows, logprobs: -8.768826, linear probability: 0.02%
Predicted token: Show, logprobs: -8.893826, linear probability: 0.01%

Look at all the results of a run: each ending with the same decimal digits for precision. Yet the top result has more digits of the same “precision”.

You see rounded impossibilities such as [-9.00, -11.00, -11.25, -12.25], followed by additional digits.

Although I haven’t seen it, a -9999 is just a probability approaching 0.

It is clear someone at OpenAI is laughing at any developer hoping to employ logprobs meaningfully now.

3

-9999 dB is an insanely low number. Basically a banned token. How frequent is this? Seems like a glitch, or default, or some internal error. Not possible to actually hit.

4

Or something added to prevent a div/0 or to short-circuit any further processing.

mpmath.exp(-9999)
mpf('3.0865651575934853e-4343')

“you’re saying there’s still a chance!”

I imagine placeholder possibility if they are running an undisclosed top-x, or other dictionary truncation algorithm on the token set.

I’d be interested to see where it appears in a top-logprobs=20 list, with others.

probability = e-9999 , not decibels…

Logprobs is also putting out other dumbness, like top-1 = 0.0 = 100% probability, and then 19 more follow.

5

But -9999 is actually called “Nepers” (since base e) and corresponds to −86,850 dB. Still, impossible to hit!

If you told anyone you saw a −86,850 dB value, they would laugh at you.

Nice Dumb and Dumber reference :rofl:

2 Likes