Thank you very much for your answer. So you recommend temperature = 0.4 and top_p = 1?
I’m also using a local Huggingface model (GPT-J) where I can set top_k. What value would you recommend for top_k in that case? The default value is 50 for top_k.
Thank you very much for your answer. So you recommend temperature = 0.4 and top_p = 1?
I’m also using a local Huggingface model (GPT-J) where I can set top_k. What value would you recommend for top_k in that case? The default value is 50 for top_k.