Logprob Value of -9999 in API Responses

API Feedback
1

Hi everyone,

I’ve noticed that in some API responses, the log probability (logprob) for a token is returned as -9999. I understand that this might be used as a marker to indicate that a valid probability wasn’t assigned, but I haven’t found any official documentation on this behavior.

Could someone clarify:

  1. Is -9999 an official or expected sentinel value in the OpenAI API for cases where no valid log probability is available?
  2. What exactly does this value signify in terms of the model’s confidence or probability assignment?

Any insights or references to official documentation would be greatly appreciated. Thanks in advance!

2

OpenAI is messing with the values returned from logprobs.

Take a look at these top logprobs showing as a result of stepping through more max_tokens of gpt-4o-mini:

Sentence: My least

Predicted token: My, logprobs: -2.9756122e-05, linear probability: 100.0%
Predicted token: favorite, logprobs: -10.87503, linear probability: 0.0%
Predicted token: "My, logprobs: -11.75003, linear probability: 0.0%
Predicted token: My, logprobs: -13.12503, linear probability: 0.0%
Predicted token: my, logprobs: -15.37503, linear probability: 0.0%

Sentence: My least favorite

Predicted token: My, logprobs: -0.00016623331, linear probability: 99.98%
Predicted token: "My, logprobs: -9.000166, linear probability: 0.01%
Predicted token: activity, logprobs: -11.000166, linear probability: 0.0%
Predicted token: my, logprobs: -11.250166, linear probability: 0.0%
Predicted token: My, logprobs: -12.250166, linear probability: 0.0%

Sentence: My least favorite TV
Predicted token: My, logprobs: -0.018826004, linear probability: 98.14%
Predicted token: show, logprobs: -4.018826, linear probability: 1.8%
Predicted token: "My, logprobs: -8.018826, linear probability: 0.03%
Predicted token: shows, logprobs: -8.768826, linear probability: 0.02%
Predicted token: Show, logprobs: -8.893826, linear probability: 0.01%

Look at all the results of a run: each ending with the same decimal digits for precision. Yet the top result has more digits of the same “precision”.

You see rounded impossibilities such as [-9.00, -11.00, -11.25, -12.25], followed by additional digits.

Although I haven’t seen it, a -9999 is just a probability approaching 0.

It is clear someone at OpenAI is laughing at any developer hoping to employ logprobs meaningfully now.

3

-9999 dB is an insanely low number. Basically a banned token. How frequent is this? Seems like a glitch, or default, or some internal error. Not possible to actually hit.

4

Or something added to prevent a div/0 or to short-circuit any further processing.

mpmath.exp(-9999)
mpf('3.0865651575934853e-4343')

“you’re saying there’s still a chance!”

I imagine placeholder possibility if they are running an undisclosed top-x, or other dictionary truncation algorithm on the token set.

I’d be interested to see where it appears in a top-logprobs=20 list, with others.

probability = e-9999 , not decibels…

Logprobs is also putting out other dumbness, like top-1 = 0.0 = 100% probability, and then 19 more follow.

5

But -9999 is actually called “Nepers” (since base e) and corresponds to −86,850 dB. Still, impossible to hit!

If you told anyone you saw a −86,850 dB value, they would laugh at you.

Nice Dumb and Dumber reference :rofl:

6

Hey @marigandarillas8 , OpenAI Support here — can you share an example of your API request body and response body? Thanks!

7

We are also seeing this issue with gpt-4o-2024-08-06 and -11-20 models. This started on March 18th for us.

We get a normal response, but a number of tokens have logprobs of -9999. The tokens are not just content, but also json formatting tokens like brackets and colons. Unfortunately we are not able to share the full request body or response in a public forum. Here is an example of one of the token responses:

{
‘token’: ‘":’,
‘bytes’: [34, 58],
‘logprob’: -9999.0,
‘top_logprobs’:
},

8

This didn’t need any special prompt - damaged output was pervasive, besides the -9999 values. Showing in this topic: Top K Log Probabilities are not aligned with actual response token - #3 by _j

Here’s is today’s response, with the major symptom fixed to align with new behavior – that returns below the top-20 will be labeled -9999 – revealing nothing deeper to you.

The values themselves are odd though - sticking some more digits of precise common imprecision on the end of very quantized values like " allet: -14.000021", athe: -14.000021 – upon close inspection, this seems to originate in the sampled token, as that 0.00000 gives all the other logprobs trailing zeroes.

And still a fault that (0.00 + the rest) > 100% probability mass.

Now on to see if logit_bias remains completely broken…

Sampled and top tokens of a poem, now:

Selected Token Logprob Top Alternatives (token, logprob)
B -1.434747 In, -0.434747
B, -1.434747
Wh, -2.934747
Am, -3.934747
Under, -3.934747
Upon, -4.684748
A, -4.934748
Moon, -6.934748
On, -7.059748
Through, -7.184748
Stars, -7.809748
Among, -8.059748
Within, -8.309748
The, -8.559748
Between, -8.684748
Beyond, -8.684748
Lost, -8.934748
Silent, -9.059748
At, -9.184748
Where, -9.184748
ene -0.000021 ene, -0.000021
ough, -12.125021
reeze, -12.375021
ubbles, -12.875021
reathe, -13.125021
ree, -13.625021
allo, -13.625021
allet, -14.000021
athe, -14.000021
asking, -14.625021
ask, -15.000021
ritt, -15.375021
ending, -16.000021
ram, -16.000021
rist, -16.125021
end, -16.250021
itter, -16.375021
ells, -16.500021
ub, -16.625021
orne, -16.625021
ath 0.000000 ath, 0.000000
volent, -23.375000
<|end|>, -23.500000
athe, -23.750000
at, -25.500000
ATH, -26.000000
ath, -26.375000
ith, -27.375000
vol, -27.500000
beneath, -27.625000
atha, -27.750000
arth, -28.625000
aths, -28.750000
ach, -28.875000
athy, -29.000000
a, -29.125000
Ath, -29.625000
-ath, -29.750000
af, -30.125000
\n, -30.250000
the -0.024139 the, -0.024139
a, -3.899139
moon, -6.399139
silver, -7.274139
twilight, -7.649139
night's, -8.399138
an, -9.649138
autumn, -9.774138
st, -9.899138
whisper, -10.024138
whispered, -10.149138
star, -10.274138
soft, -10.649138
pale, -10.774138
velvet, -10.899138
whispers, -10.899138
midnight, -11.149138
night, -11.399138
cer, -11.399138
silent, -11.649138
ancient -4.801350 silver, -0.676350
moon, -1.426350
whisper, -2.426350
twilight, -3.551350
velvet, -3.801350
willow, -4.551350
ancient, -4.801350
silent, -5.051350
whispered, -5.176350
whispers, -5.176350
quiet, -5.551350
midnight, -5.551350
canopy, -5.551350
veil, -5.551350
stars, -5.676350
cres, -5.676350
sil, -5.801350
azure, -5.926350
sapphire, -5.926350
w, -6.051350
o -4.253455 willow, -0.378455
oak, -1.878455
,, -3.378455
whisper, -3.628455
canopy, -3.628455
b, -4.128455
o, -4.253455
moon, -4.878455
trees, -5.253455
maple, -5.378455
forest, -5.878455
elm, -5.878455
twilight, -5.878455
tree, -6.003455
silver, -6.003455
bany, -6.503455
whispers, -6.503455
e, -6.753455
cedar, -6.878455
sky, -7.128455
aks -0.061979 aks, -0.061979
aken, -2.811980
akt, -11.686979
aky, -13.061979
akw, -14.561979
aths, -15.186979
aking, -15.936979
akh, -16.061979
aked, -16.436979
ak, -16.561979
akse, -17.186979
ar, -17.561979
aker, -17.561979
akin, -17.561979
'er, -17.686979
ake, -17.936979
<|end|>, -18.936979
des, -19.186979
ars, -19.311979
aksan, -19.311979
, -0.876790 ,, -0.876790
where, -1.001790
', -1.876790
in, -4.251790
that, -4.626790
we, -4.751790
they, -5.001790
’, -5.126790
a, -5.626790
with, -5.876790
so, -6.001790
of, -6.126790
I, -6.626790
whispers, -6.751790
the, -7.126790
whisper, -7.251790
at, -7.501790
their, -7.876790
entw, -8.001790
and, -8.126790
where -1.064168 where, -1.064168
whispers, -1.189168
the, -1.439168
a, -3.189168
shadows, -3.689168
in, -4.064168
time, -4.939168
secrets, -5.064168
dreams, -6.064168
moon, -6.064168
their, -6.189168
whispered, -6.189168
whisper, -6.689168
twilight, -6.814168
stars, -7.189168
with, -7.439168
we, -7.564168
silence, -7.814168
soft, -7.939168
I, -8.439168
time -4.978883 shadows, -0.228883
whispers, -1.728883
secrets, -4.728883
time, -4.978883
silence, -5.228883
twilight, -6.228883
dreams, -6.728883
moon, -6.978883
stories, -8.228883
sunlight, -8.228883
the, -8.353883
echoes, -8.853883
whisper, -9.103883
whispered, -9.228883
night, -9.353883
light, -9.478883
wild, -9.478883
dusk, -9.728883
sun, -9.853883
quiet, -9.853883
stands -1.291720 stands, -1.291720
forget, -2.041720
does, -2.291720
whispers, -2.541720
is, -3.041720
forgot, -3.166720
lies, -3.666720
unw, -3.791720
susp, -3.916720
unfolds, -3.916720
sleeps, -4.041720
's, -4.291720
softly, -4.291720
breath, -4.416720
holds, -4.416720
slows, -4.416720
dr, -4.541720
we, -4.666720
grows, -4.916720
lays, -4.916720
still -0.003416 still, -0.003416
hus, -6.878417
shy, -7.378417
slow, -7.628417
tall, -7.878417
mute, -9.378416
sly, -9.378416
free, -9.503416
st, -10.378416
thin, -10.378416
fra, -10.503416
eer, -10.753416
lost, -10.878416
fleet, -10.878416
quiet, -11.003416
near, -11.128416
sweet, -11.128416
meek, -11.128416
n, -11.253416
ve, -11.253416
, -0.002537 ,, -0.002537
and, -6.002538
in, -10.877538
anew, -11.127538
to, -11.877538
as, -13.002538
\n, -13.002538
so, -13.127538
with, -13.752538
—, -13.877538
wide, -13.877538
,\\, -13.877538
ag, -14.002538
,\\\n, -14.002538
,\n, -14.127538
's, -14.127538
., -14.377538
today, -14.377538
alone, -14.377538
at, -14.502538
\n -0.000436 \n, -0.000436
\n, -7.750436
\n, -12.750436
\\\n, -13.500436
, -15.375436
a, -15.875436
\\\n, -16.375437
\r\n, -16.375437
the, -16.875437
\n, -16.875437
\n \n, -17.125437
, -17.250437
⠀, -17.250437
we, -17.500437
\r, -17.750437
<|end|>, -17.750437
, -17.875437
, -17.875437
⁠, -17.875437
, -18.250437
Wh -0.272977 Wh, -0.272977
A, -1.897977
The, -3.272977
Moon, -3.522977
Soft, -5.647977
Dream, -5.897977
D, -6.147977
In, -6.397977
Golden, -6.397977
Silent, -6.522977
Silver, -6.647977
Echo, -7.147977
L, -7.272977
Leaves, -7.397977
Stars, -7.772977
Gent, -7.772977
Wis, -7.897977
Sil, -8.022977
Sh, -8.147977
Dance, -8.272977
ispers -0.029754 ispers, -0.029754
isper, -3.529754
isk, -13.529755
istles, -13.654755
is, -15.279755
isp, -15.279755
ist, -15.529755
ims, -15.529755
im, -15.904755
whispers, -15.904755
irls, -16.529755
ose, -17.529755
iskey, -17.529755
irling, -17.654755
irl, -17.779755
ither, -18.279755
ip, -18.404755
istle, -18.654755
ir, -18.904755
ence, -19.029755
of -0.035507 of, -0.035507
dance, -3.785506
weave, -4.785507
in, -6.660507
ride, -7.285507
from, -8.410506
trace, -8.785506
w, -8.910506
float, -9.035506
echo, -9.160506
gather, -9.410506
on, -9.535506
cradle, -9.535506
drift, -9.660506
tw, -9.785506
thread, -9.910506
carry, -10.035506
blend, -10.160506
linger, -10.160506
entw, -10.160506
forgotten -1.635978 the, -0.510978
forgotten, -1.635978
moon, -3.510978
dreams, -3.635978
twilight, -3.635978
secrets, -4.010978
old, -4.635978
leaves, -4.635978
autumn, -4.635978
lost, -4.760978
wind, -4.885978
yesterday, -5.010978
y, -5.260978
stories, -5.260978
dusk, -5.260978
a, -5.385978
stars, -5.385978
past, -5.635978
shadows, -5.635978
dawn, -5.885978
dreams -0.284643 dreams, -0.284643
tales, -1.534643
winds, -4.409643
lore, -5.034643
worlds, -5.784643
stories, -5.909643
songs, -6.159643
leaves, -6.409643
secrets, -7.159643
days, -7.534643
stars, -7.534643
myths, -7.784643
legends, -8.659643
bree, -8.784643
realms, -8.784643
streams, -8.909643
echoes, -9.409643
lives, -9.534643
woods, -9.784643
ages, -9.909643
are -9999.000000 softly, -1.518961
dance, -1.893961
gently, -1.893961
in, -2.018961
weave, -2.393961
begin, -2.768961
entw, -3.643961
the, -3.768961
on, -3.893961
w, -4.518961
take, -4.518961
drift, -4.518961
ride, -5.018961
float, -5.268961
linger, -5.268961
ca, -5.393961
embrace, -5.393961
,, -5.643961
quietly, -5.643961
echo, -5.768961