Pricing when requesting logprobs to calculate Perplexity Score

Logprobs are simply part of the API response if you turn them on with the two required parameters, and do not cost anything. They do not take extra computation (except for OpenAI’s filtered version not being what the sampler actually uses but a different calculation that misrepresents the chance of special tokens that might invoke tools or stops.)

Wherever you read that information is incorrect.

AI is powerless to resist a leading question with falsehoods, and produces mere justification:

1 Like