Does OpenAI have a top_k parameter?

[martin.erca.2017]

Hello,

Does OpenAI have a top_k parameter?

If yes, what is the default value and range of top_k?

Thank you.

[dignity_for_all]

As far as I understand, there seems to be no language model currently provided by OpenAI that allows you to specify the top_k hyperparameter.

Top-K sampling considers the top k tokens, but you can use the Top-p (nucleus) sampling value as a hyperparameter.

In language models, top_p and top_k are not mutually exclusive concepts, but it seems that only top_p is available as a hyperparameter through the API at the moment.

It seems that some past language models provided top_k sampling as well.Please take this into consideration too.

Top K? How can I set it?

I understand that the API allows us to specify for Top P, but how about Top K? Is there a default Top K that is already implemented? Or can we change Top K by deriving it from Top P?

[tocsa]

Do we know the reason for this? Is it possible that top_k will be introduced on the API? When someone wants to decrease chaos, - besides setting temperature to 0 - it’s somewhat easier to set the top_k to 1 than guessing a top_p. Everywhere else top_k is a standard knob, why would the OpenAI API leave it out if temperature and top_p is already available? I’m kind of shocked.