N argument vs batch input

API
1

Hello, I am doing something similar to self-consistency paper [2203.11171] Self-Consistency Improves Chain of Thought Reasoning in Language Models and wondering that what is the difference between

  • n argument in the API call, and
  • batch input in OpenAI API by sending a list of prompt instead of single prompt

Thanks!

2

You are asking, if I understand you correctly, please correct if not, you wish to know the difference between completion params and fine-tune params?

  • n is a completion param

  • batch_size is a fine-tuning param

Could you be more exact @v1l2xsvqe regarding the OpenAI API params you are interested in?

Thanks!

:slight_smile:

3

N argument tells gpt to send you back n times the number of completions for a single prompt

Batch input means you give gpt multiple prompts and it gives you a completion per prompt

If you pass the same prompt multiple times in a batch it may give you the same completion for each of them (depending on your temperature setting)

But if you use n, the chances are that you will get different answers (also depends on temperature)