Hello,
Does OpenAI have a top_k parameter?
If yes, what is the default value and range of top_k?
Thank you.
As far as I understand, there seems to be no language model currently provided by OpenAI that allows you to specify the top_k hyperparameter.
Top-K sampling considers the top k tokens, but you can use the Top-p (nucleus) sampling value as a hyperparameter.
In language models, top_p and top_k are not mutually exclusive concepts, but it seems that only top_p is available as a hyperparameter through the API at the moment.
It seems that some past language models provided top_k sampling as well.Please take this into consideration too.
2 Likes