I am using gpt-4o-2024-08-06 and extracting the token top log probabilities. However, the outputs are incorrect. For some tokens, gpt4o directly copies the token probs from the previous token.
For example, for tokens [“co”, “efficient”], the output looks like the following:
token co
Logprobs data: [TopLogprob(token=‘co’, bytes=[99, 111], logprob=0.0), TopLogprob(token=‘cor’, bytes=[99, 111, 114], logprob=-19.75), TopLogprob(token=‘coeff’, bytes=[99, 111, 101, 102, 102], logprob=-20.625)]
token efficient
Logprobs data: [TopLogprob(token=‘co’, bytes=[99, 111], logprob=0.0), TopLogprob(token=‘cor’, bytes=[99, 111, 114], logprob=-19.75), TopLogprob(token=‘coeff’, bytes=[99, 111, 101, 102, 102], logprob=-20.625)]
The top log probs for “efficient” is completely off. The output token “efficient” is not even in the top list, and the top logprob list is identical to the previous token “co”.