Is there a way to use the API to see the probability of existing tokens without generating more tokens?
For example, I want to input something like
Q: How many legs does a dog have?
A: 10
And see the probability that the model generated “10”
You can do this in the playground, so wondering if there’s an API that supports this as well
Yes, you can use the API to see the probability of existing tokens by setting ‘echo=True’ and by setting the log probs parameter when you make a Completion call. We have documentation for this in our API Reference.
One thing to note is that you do need to submit a completion call, even in the Playground. You can only look at the logprobs for the prompt after you generate a completion in the Playground. This is enabled by that ‘echo’ parameter.
@jforte that’s great that the Completion endpoint offers this. Is there a way to do this with the ChatCompletion endpoint? This would be super useful for probabilistic reasoning.