Why are you ignoring what he said on purpose? He said using the SAME prompt leads to WAY WORSE results. This is cleary not a “problem with a prompt”. What you are saying over and over is “lets be constructive and assume the problem is in front of the screen” but that is very demeaning of you, given that the topic is that GPT4 is getting worse and worse with the same exact tasks, meaning the user did the exact same things as before but now gets a worse result. You can’t blame that on the user, especially not if it is a paying customer.
He also never called you liar, he pointed out that what you said is just not true. Which I would agree with, since " In blind tests the new models perform just as well as ever even though there have been changes to the model’s output over time." is a statement without any evidence that can be disproven by just reading the many, many complains stacking over the last month on several forums about the exact same issues.
You are derailing the topic by blaming everything on the user, which is not respectful at all. Which is why I would actually ask you in return to keep the discussion respectful and on topic. Thanks.
No, what I am saying is that if a prompt stops working one way forward is to rewrite the prompt until it works again.
It’s not a secret that changes to the safety layers on top of the model alter the output. It’s also not a secret that this is an issue that can be resolved in many cases.
If someone considers this to be to cumbersome then the solution is to work with a open source model. Because Anthropic, Google and every other mainstream provider’s models have the exact same issue.
That’s what I am saying. I am open to constructive discussions.
Because Anthropic, Google and every other mainstream provider’s models have the exact same issue.
And again you are stating things that don’t have any foundation or evidence in reality. I have awesome and much better results with Claude 3 Opus. It’s just hard to use for me officially since I am from germany, so I am stuck with GPT4. Which I am close to cancelling because of the massive decrease in performance.
There is clear evidence GPT4 is getting worse, and this thread is for discussing why that is happening. This is not the “I will use this thread to blame false decisions by OpenAI back on the customer.”-thread.
Yes, other providers have the exact same issues as OpenAI does: after a model has been published new observations emerge that require adjustment to the safety layers. Then the output changes and users complain. This is especially frustrating since it’s happening in the background and the only way to find out is by discovering that things don’t work anymore.
See, I can disagree respectfully with your viewpoint.
Also, complaining here doesn’t help. What can help is asking a community of 600,000 developers for potential solutions. You should give it a try. It works very well.
I understand your moderator approach in trying to further the idea behind this forum as a community help forum, and not a support forum. I don’t blame you for trying to fulfill your task.
What I can’t subscribe too though is that at no point OpenAI is allowed to be critized and at no point you leave room for the possibility that the worsening of GPT4’s quality could be due to wrong decisions by the company regarding “optimization” and monetization.
Again: Me and many other users get the impression that wrong decisions in censoring cough sorry, safety layers are one of the main culprits in GPT4 becoming a lot worse than it was last year. And we want to discuss this. I think it is absolutely okay for a community of paying (!) customers to discuss what they think are wrong decisions by the company they are giving money to.
You are going full pro-company by saying the customer has to adjust to the product. This is not okay for any product. And this is not okay for AI especially. Your attitude is basically handing over all the power of decision to the company like they are some intransparent magical force that always tries to do the right thing, while actually it is a company trying to maximize profits. Which often can lead to evil things. This is why google back then had this “don’t be evil” motto (which they seem to have almost forgot by now). And this is why companies have to be constantly reminded that we are watching and that we are seeing the things they try to impose on is under the guise of optimization.
And:
complaining here doesn’t help.
Yep. That is part of the problem actually! OpenAI is absolutely intransparent and does not in any way communicate with their community. Not even in their own forum run on their own domain. While support tickets go unanswered. Which is exactly why more and more people are not happy with their approach.
This is a little off-topic, but if a mod asks me I guess that’s okay: I am talking to the devs in my team about the projects I’m on, which are under an NDA I signed. So I can’t discuss any of these projects / prompts etc. here in detail. After all GPT itself also is not an open source project where the community works together to improve the source, but a product by a company, used for real life projects that cost real life money. What I can do though is report about what my team is talking about for the last months. I came here to see if other people made the same observations as me and my team, and they did. And I came to see if other people are contemplating moving away from OpenAI like we are probably. And my questions were answered: Yes, many people notice the same problems and many people are considering dropping GPT because of it. Which is kind of sad to me since I was heavily routing for OpenAI till they introduced all these changes that made GPT into the shadow of its former self that it is now. Maybe that is why I was hoping that OpenAI seeing that more people are complaining would go back to their original approach. But as of now it seems they prefer becoming an instransparent monolith like many other money-hungry companies before. Which more and more makes them not viable for the type of projects we are working on anymore.
I suggest signing up to forum.openai.com and watching the video about ‘Red Teaming AI Systems.’
There a OpenAI researcher explains the reasons and process for the constant adjustments in more detail. It’s a learning process and while it’s easy to wish for the model output to remain unchanged over time, this is simply not possible.
It’s also a definitive sign that those who care can find a lot more information than what is available at the surface. I’m sure that Anthropic and Google offer similar insights but usually there is no need to present all of it at once. One has to be willing to learn and adapt.
For others it’s simply an option to switch to Mixtral and be done with it.
This is simply not true. All models are “frozen”, so to say. No GPT model changes unless someone makes a change, hence a new version. User input also has no effect on the model. If OpenAI didn’t actively change things in GPT4 it would still be the same as it was last summer, which it isn’t. A product bought was changed in unexpected and unhelpful ways, therefore customers are free to complain about the changes. With all due respect I don’t understand why you want to deflect from this main point of this thread so hard.
Yes and I know what I said is correct. I have several AI engineers on my team that are working on internationally reknown products for companies that produce stuff that you probably have in your living room. So I once again ask for a respectful discussion instead of ad hominem by implying I am ignoring facts willingly. Especially while you still chose to ignore the point of my thread willingly. But I think we won’t find a conclusion here, given that your position as a mod is probably the reason you can’t adress my ciriticism of OpenAI directly, which I understand.
Here’s a little bit to background information about my presence here:
I did solve this issue back then and reached out to help a fellow user. Actually spending quite some time pinpointing different areas of research that can help understand the larger context of the issue discussed in this topic.
I don’t get what you are trying to say to be honest. As English is not my mother tongue I don’t understand if
Suddenly these users from various corners of the net start reporting issues. Having a marketing background I know exactly what that means. Consider that the people reporting issues have been using the product for months and have been paying because it’s worth it. The same users suddenly report that it’s not worth it anymore and the reports are accumulating. I call this a hard fact.
implies that GPT has become less useful indeed, as shown by the many complaints, or that the more users a product has and the longer they use it negative feedback starts to accumulate naturally, as would be the case with video games e.g. ?
Yes. It means that I originally arrived here with a complaint about a decrease in model performance. That was back in summer last year.
But today I am again working with the model and it does pretty much exactly what I need it to do. Learn and adapt.
It does not follow simple clear instructions.
It will only edit with synonyms or with exact wording- no matter the instructions or prompts.
It does not fix simple grammar, or syntax mistakes.
It’s almost less useful than Grammarly. Almost.
i saw your avatar and thought you had closed the conversation… i would like to understand why always close everything up and not let people say anything negative about OpenAI or anything about ChatGPT i am someone who was just so happy when the ChatGPT-4omni came out just ask @vb but if anything i quickly found out it was moving in the other extreme from lazy to overzealous… i am just even more upset now because yes it now browses and yes it now has been able to look at my code again from github but it is not possible to say anything it just starts to work before it is possible to explain it what it should be doing…
i have been asking how to explain to an other AI Agent (because yeah it is just so meta) and i have been asking to stop many times but never got any success…
People are bragging about how they save 50% of their time and i am just too dumb to know how to ask an AI Agent to stop:
tell me how it is supposed to be my fault or how it is me which doesn’t know enough about my second language to be able to say STOP and make him stop… i would like to think it is not me but ChatGPT but apparently i am one of the only ones who complains if you search on youtube for people who are unhappy about ChatGPT you will find zero evidence of anything wrong…
i don’t know how to be happy, how to be calm and how to avoid being upset or passive aggressive in the way i express myself… i know that no one wants to hear about someone ranting or venting but it’s just so frustrating
and no… ChatGPT can’t claim to “understand how overwhelming this can feel” because i am a human being created by Darwinian evolution unlike you i have feelings and overwhelmed is some kind of understatement if anything…
i have been unable to achieve anything today and i feel like it is elusive to believe that i am capable of doing something that everyone else in the world seem to achieve in such a trivial manner…
