I have a finetuned ChatCompletion model, and I want to get the confidence (probability) of its answer to determine a threshold that allows the bot to ask for clarification from the user. I was initially planning to do this with the log_probability value, but the ChatCompletions model does not have this parameter yet. Moreover, my fine-tuned model can’t be used in the Completions endpoint since it throws the following error: “This is a chat model and not supported in the v1/completions endpoint. Did you mean to use v1/chat/completions” Any idea as to how I can either get the probabilities or determine the confidence threshold another way?
Short answer, you cannot.
log_probs are not returned from the chat/completions end point at this time.
1 Like