I’m currently working on developing a chatbot application that utilizes a fine-tuned GPT-3.5 Turbo 0613 model. I was wondering if anyone could kindly share their insights on the optimal settings for temperature and top_p to ensure the chatbot exhibits a more human-like behavior while adhering to the provided system prompt. I’ve come across suggestions suggesting a temperature of 0.5 and a top_p of 0.5 for the chatbot, but I’ve noticed that these values tend to make the chatbot repeat the same questions, which can be a bit frustrating for the user. Any advice or recommendations would be greatly appreciated. Thank you in advance!
The AI will only repeat the same questions if the user repeats the same questions…
with top_p nucleus sampling parameter low to decrease the tail of strange tokens, you can increase temperature higher and still have creative selection of language paths emerge.
The default top_p and temperature seems much better, do you know the default top_p and temperature of openai.ChatCompletion.create()? I’d like to adjust the parameters from their default settings.
The defaults for both are 1. The defaults are not the best for language though, that is up to experimentation for the particular model, and particularly must be lowered for today’s high-perplexity gpt-3.5-turbo if it will be exposed to non-English.