See probability of existing tokens

API
1

Is there a way to use the API to see the probability of existing tokens without generating more tokens?

For example, I want to input something like

Q: How many legs does a dog have?
A: 10

And see the probability that the model generated “10”

You can do this in the playground, so wondering if there’s an API that supports this as well
image

1 Like
2

Yes, you can use the API to see the probability of existing tokens by setting ‘echo=True’ and by setting the log probs parameter when you make a Completion call. We have documentation for this in our API Reference.

One thing to note is that you do need to submit a completion call, even in the Playground. You can only look at the logprobs for the prompt after you generate a completion in the Playground. This is enabled by that ‘echo’ parameter.

3

@jforte that’s great that the Completion endpoint offers this. Is there a way to do this with the ChatCompletion endpoint? This would be super useful for probabilistic reasoning.

2 Likes