GPT-4o not following simple and clear instructions

I am asking GPT-4o to write a paragraph between 400 and 500 words. It will not follow this instruction and wrote the first paragraph with 635 words (counting using google docs word count). I ask it to do it again with a word count between 400 and 500, and it proceeds to send an almost identical paragraph and tell me it is 499 words, when in reality it is 621. I asked a third time and it sent the exact same thing except saying it is 471 words, whilst once again it is 621. GPT-3.5 has no issue doing this and I would like to know what the point is in providing a new GPT when it is absolutely terrible and can’t do anything I ask of it that previous versions could.

4 Likes

Models are by default not good at or capable of counting. So it is a well-known constraint that you cannot expect a model to produce an exact word count, even if you include that as an instruction into your prompt.

That issue is consistent across models, i.e. when I ask a gpt-3.5 model to produce a 500-word story, the final word count will also not be 500.

Some developers have resorted to workarounds, whereby they programmatically count the output of an API response and then try to make some course corrections in a follow-up API call.

1 Like

I completely agree with your experience regarding GPT-4o not following instructions properly.

I’ve run multiple tests and confirmed that the issue isn’t with my prompts either. GPT-4o consistently fails to adhere to specific instructions, much like how you described.

This situation is reminiscent of the earlier issues with GPT-4, where users reported the model being “lazy.” Initially, there was criticism suggesting the problem was with the users’ prompts. However, OpenAI eventually acknowledged the issue and made necessary fixes. I believe a similar pattern is emerging with GPT-4o.

At the moment, due to this consistent failure to follow simple instructions, GPT-4o is not usable for me either. I’ve reverted to using GPT-4-turbo for reliable performance.

I hope OpenAI addresses this issue soon, and until then, I’ll continue using the alternative.

4 Likes

I confirm that “GPT-4o” doesn’t follow any instructions.

@jr.2509 , it’s not about counting. It’s not following the instructions from " Customize ChatGPT":

“GPT-4” doesn’t have such problem and follows custom instruction and requested response style without any problems. I could live with that by using “GPT-4” but the problem is that now all custom GPTs have automatically been switched to the “GPT-4o”. And they all stopped following the custom instructions.

This is a big issue.
You may think of a workaround when you add text from custom instructions to the GPT configuration, but this is a bad idea, when the GPT is public and there might be some private information in the custom instructions.
And at the end of the day adding the same custom instructions text over and over again to all the GPTs doesn’t look like a good solution.

1 Like

We’re also having issue with GPT-4o not following system instructions. We’ve seen a steady decline in GPT model ability to follow instructions. GPT-4 was wonderful. GPT-4-turbo was worse. And GPT-4o is either on par with GPT-4-Turbo or worse.

I appreciate the increased power of the new model version, including multi-modality, later knowledge cut-off dates, higher rate limits etc, but how well the model does what you want is THE THING that we all need as a baseline to use these models.

4 Likes

Yup. This seems like a BIG problem. I just discovered one of my custom GPTs is badly broken, and I assume it’s because it’s using 4o now.

It used to follow my clear step-by-step instructions really well. Now it seems to randomy decide which steps to follow, and often just jumps to the last step - which is a step it’s not even meant to do unless requested by the user.

Very frustrating.

And there’s no way to specify a GPT should use GPT-4 is there?

1 Like

Same issue here. GPT 4o is returning terrible inconsistencies and no matter how many times I rewrite it, the results are exactly what I don’t want. Give us the option to choose the output model. GPT-4 is the best by far.

1 Like

Not just counting bro. But ChatGPT 4o is the worst model in following any kind of instructions. It just says what it wants to say. Does not follow or change the conversation.
I used to use GPT 4 for programming help but this model just keeps repeating mistakes while there is clear instructions to not do so or to not talk about a specific thing. But it just does. They have also reduced the limit on GPT 4 since its release and honestly this is even worse than 3.5 model!

2 Likes

I have also reported worse ability to follow instructions. I had to revert back to GPT-4-turbo. Also the latency was worseded as well.

1 Like

Right now, 3.5 works. 4 and 4o isn’t working at all.

I have to ask it like 20 times for it to follow instructions. They are clear instructions.

“write in paragraphs. no lists.”

Writes lists.

Like for real?

1 Like