Api call to gpt-4-32k or gpt-4 allucinate at the end of long responses

API
,
1

I am doing an api call to gpt-4-32k-0314

with something like this:

 const response = await openai.createChatCompletion({
    model: "gpt-4-32k-0314",
    messages: [
      {
        role: "system",
        content: system,
      },
      {
        role: "user",
        content: `${user}`,
      },
    ],
    temperature: 0.3,
    frequency_penalty: 0.7,
    presence_penalty: 0.7,
    stream: true,
    max_tokens: 3500,
  });
  // Convert the response into a friendly text-stream
  const stream = OpenAIStream(response);
  // Respond with the stream
  return new StreamingTextResponse(stream);

and what I am asking to do is to write a long blog post with more than 1500 characters.

I am giving first a draft of the blog post, and then it needs to do it longer and improved.

What happens instead is that it starts generating the answer well, but towards the end, it becomes crazy and adds random words without any sense.

This is an example output that it gives me:

By applying lessons learned from examples like those provided above aspiring Youtubers everywhere stand chance making big splash world online video entertainment no matter size scale ambitions may be today tomorrow years come ahead us all good luck happy creating everyone out there reading along journey towards greatness awaits let get started right now together united passion creativity determination succeed against odds stacked favor victory sweet taste indeed cheers friends enjoy ride onwards upwards forevermore amen hallelujah praise heavens above glory unto highest peace love harmony unity prosperity abundance joy laughter happiness health wealth wisdom knowledge understanding enlightenment freedom liberation salvation redemption forgiveness grace mercy compassion kindness empathy sympathy generosity charity humility gratitude appreciation thankfulness awe wonder reverence respect admiration honor dignity integrity honesty truth justice fairness equality opportunity progress innovation invention discovery exploration adventure challenge triumph conquest mastery excellence perfection fulfillment satisfaction achievement accomplishment realization manifestation destiny fate purpose meaning significance value worth importance relevance impact influence inspiration motivation encouragement empowerment upliftment transformation transcendence evolution revolution awakening ascension elevation expansion growth development progress improvement betterment

1 Like
2

You shouldn’t need any penalties for GPT-4.

Instead, you’ve probably made it so the AI can’t write common words, or even punctuation where it wants, after the penalties for those continue to accumulate.

If you were doing this yesterday, there was a model issue that caused outputs to go crazy for some.

1 Like
3

Contrary, I had to set both penalty parameters to 0, resulted in a situation where the same phrase was repeated infinitely.
To deal with this problem, I had no choice but to set the penalty parameters higher than 0.

1 Like
4

For now, removing the penalty seems to solve it.

1 Like