It seems it takes a very low top-p parameter to actually constrain the logits in cases where there is some leeway of choice.
At temperature 2, using some very indeterminate leading language, I get sentences that are mostly from the top three when using a top-p 0.05
These low and similar percentages are of course a contrast to something like “training an AI using few-”, which gives 99.99% of five different ways of writing “shot”, with the top choice at 98%.
(note: model probabilities are returned before temperature and nucleus sampling, so we don’t get to see the changing distances of probabilities or the limited token choices)
So that makes a high temperature above 1 but quite low top-p an interesting case for unexpected writing, creative without going off-the-rails crazy, almost randomly picking from the few top choices when choices are made available.
So for example at temperature 2, top-p 0.1, it is only in line four of our poem that we finally get two second-place choices (here using untrained davinci GPT-3)
Temperate of 2 would otherwise produce nonsense.
Of course picking anything but the best is completely unsuitable for code or function-calling.