Finetuned GPT-3 Repeating responses

API
1

I have finetuned one Curie model for my usecase. Now whenver I’m making the API call, It is not stopping after providing the response. It starts repeating the sequence again until all the max_tokens is consumed.

A sample response looks like this
Ques: A sample question

Sample response

END

Sample Response Again

END

Sample Response Again

END
.
.
.

This goes on until all the tokens are consumed. I can strip from the first END, but I’m loosing the cost with unnecessary generation over here.

Has anyone ever faced this. If so Let me know how can I fix this. Help highly appreciated!

2

My bad, I forgot to add the stop sequence in the API call!! :sweat_smile:

For anyone who stumble upon this post. And is in the similar situation

You need to add the stop sequence in the API call. Something like this

response = openai.Completion.create(
  model="FINETUNED_MODEL_NAME",
  prompt=text,  
  temperature=1,
  max_tokens=512,
  top_p=1,
  frequency_penalty=0,
  presence_penalty=0,
  stop=["END", " END***"]
)

I have added END as I used it in training examples. You can replace it with Anything that you have applied in the training dataset.

1 Like