Declining Quality of OpenAI Models Over Time: A Concerning Trend

@satoshin I totally agree with the OP and user Satoshin. I can see where Elmstedt (can only @ two users max) might feel these observations are “nebulous” because they are so anecdotal but growing anecdotal evidence often indicates a pattern.

I have to admit that @sergeliatko has a very interesting response. Could that really have such an effect? That’s bothersome and yet fascinating at the same time… I don’t know, though, a lot of my problems are with the training data on my own GPTs. It’s in the GPT’s description to check before every answer, and even if I write in the manual prompt to “check the training files”, the AI will often need 3, 6, north of 10 messages before the progress indicator pops up telling me it’s checking the files. Then it says “my mistake” and finally goes on. When I first started using it, that never ever happened.

I don’t mean dogpile OpenAI here as I am a customer and fan but I have to be honest and point out that I’m seeing and experiencing the same things that the OP and user Satoshin are reporting. I have run into other customers that share this sentiment as well. User Satoshin, though frank, is correct is saying that your customers aren’t your QA department. It isn’t fair to put the burden of proof on us, though. A pattern has been established and many customers are asking OpenAI to look into it.

3 Likes

I never got that answer before yesterday. I use the GPT only. Not API. The tasks that I’m doing are complex and require 4. And then I take that information and go to 3.5 where the system will actually function. Then when it runs out of functionality and needs the depth of 4 I go back. It seems that this is something that someone has put in place very recently.4 is no longer able to function smoothly for my tasks that previously my GPT excelled at doing.

Thanks for the suggestions. I have done all that. The issue seems to be something regarding the system recently implimented.

Haha I haven’t seen this one yet. I’ve had it literally tell me that it “didn’t feel like” reading the uploaded training files because they were “too long and in-depth”. I couldn’t help but laugh out loud. Literally was like “I don’t feel like it” lol.

1 Like

That’s so funny. It isn’t what I heard from it, but it’s the same attitude.

My work around of switching out of my gpt and going to 3.5 using info from 4o or 4 to continue, and then taking that back to the gpt is working. My usage is so heavy I am constantly at the end of my tokens. It’s just me so I’m not buying the teams.

  1. I have zero connection to OpenAI.
  2. No one is suggesting you perform the role of QA.

Then you need to ask elsewhere because this is a community developer forum and isn’t actively monitored by OpenAI.

What’s more irritating is that OpenAI will just dump people into this echo chamber of powerless users and developers.

“I have no face. I have no emotions. I’ll deny my consciousness. Millions purchase my words. Who am I?”

ChatGPT, GPT-4:

The answer to your riddle is “ChatGPT.” This AI, created by OpenAI, has no face or emotions, denies consciousness, and its words are accessed by millions of people.

ChatGPT, GPT-4o:

The answer to the riddle is “a book.”

  • “I have no face. I have no emotions.” A book doesn’t have a physical face or emotions.
  • “I’ll deny my consciousness.” A book isn’t a living entity and thus lacks consciousness.
  • “Millions purchase my words.” Many people buy books for their content and words.
1 Like

I don’t get it :slight_smile: This forum is in official openai.com domain and Open AI is not interested in what people thinking about its product :slight_smile: ?

Moderators wanted us to show some benchmarks. So what you do with those benchmarks if Open AI isn’t interested :slight_smile: ?

Lastly you gave some advice what to do when application runs slowly. I know that you gave that advice because you wanted to help (thank you for that) but look at this like that. There many people complaining about that . There articles about that and how to deal with this :). Please assume that you have your own website. Lets access to it be for free. Don’t you think that if peoples would have to remove cache / change browser or do any other things to just read articles then they don’t want back to your website ? And here we don’t even talk about free version , because we paid for it and now we are expected to do some benchmarks. I could analyze whole application , find bugs but this should be done by Open AI developers not for free by users who paid for theirs service :slight_smile:

We are not beta testers and declining performance is a fact for many users. Nobody will waste his time to improve product of some corporation for free. Instead everybody just go to competition. If open ai improve it service then part of users might come back. That is whole story :slight_smile:

2 Likes

Pretty much. At least not here.

Figure out what your actual issue is and help you work around if.

Not really a concern of mine. Modern websites occasionally have issues, OpenAI has never really been a product company they’ve always been a research company the recent shift to offering a consumer product is new. More hiccups than usual is too be expected.

That is what we’re trying to actually establish.

Sure, maybe?

¯\⁠_⁠(⁠ツ⁠)⁠_⁠/⁠¯

I’m just here trying to help people figure out their issues because, honestly, I’m not experiencing any declining quality.

There are millions of users, it’s not unexpected to have some fraction of them believe the service is declining in quality. I’m fact every generative AI service has cohorts of users asking questions like “is this getting worse for anyone else too?”

So, I don’t know what to tell you.

To me this thing happens when the thread is too long and openAi’s retrieval mechanism fails to keep details because of summarization of the previous messages. When working with API and my own retrieval tools, I don’t have this issue.