It seems that the exact same code with logit bias settings was working one month ago but is not affecting the output now. (I set some tokens to be 100, the output is not influenced by this now.) Is that because of my openai version? Anyone have this issue as well?
Which model are you using?
Have you confirmed this with a minimal example?
I noticed that the logit_bias can still influence the output but it won’t influence the log probability of the token now. It seems that OpenAI made a change on log probabilities on March 3rd. logit_bias parameter will now only influence the sampling behavior but it will no longer change the numerical values of the returned log probabilities. Because I expect to set some tokens to be 100 in logit_bias so that the top log prob will only keep track of those tokens. Any suggestions of getting around this? I am thinking of keeping track of those token id directly but the API just allows to access most likely token prob.
1 Like