I’ve noticed a significant drop in the context awareness when generating python code. For example when I ask chat to modify the script based on my guidelines and then ask to add some functionality, it forgets it’s own modifications and alters first version of code. What is even worse, it doesn’t follow my very simple and basic instructions and instead goes wild off. And this happens on a discussion which is 6,696 tokens long with code being only 25-35 lines. It’s worse than GPT3.5. I tried multiple chats on the same topic and it’s getting even worse than that. Anyone experienced same issues over the last few days?
31 Likes
it’s possible than the quality drop might be linked to peak demand.
3 Likes
how do you figure? I imagine it either works, works slowly (memory bandwidth limits reached), or not at all (OOM) 
unless they actually deploy smaller models during peak demand without telling anyone. That would be funny as hell.
2 Likes
My experience is that all kinds of weird problems can happen when a service is overloaded
3 Likes
100% sure that quality drop a lot. I guess they are doing any update
1 Like
I am experiencing better performance combining GitHub Co-pilot and ChatGPT.
The file context awareness helps my workflow a lot and I can diff changes.
Yeah, I am feeling the same here. My assistants got dumber and dumber.
11 Likes
I’m feeling it too. I was in a conversation about a script. Something wasn’t working, fixed a mistake (context related, so not GPTs fault), fed the new script to the conversation and asked for some additions. Much to my surprise, the GPT made the changes to the script it wrote before and ignored the new script I gave it in the same conversation.
I also ended up in a loop with an error I was having. The first suggestion from GPT didn’t work, so I asked for another solution. That also didn’t work and when I let GPT know, it’s response was the first solution it gave. When I pointed that out, it apologised and then gave me the second solution again. Just for testing purposes, I kept asking for working solutions and the GPT kept alternating between solutions 1 and 2.
Never has that (at least not to that extent) with GPT4. GPT4 would at least recommend me to contact an expert when it was out of ideas.
9 Likes
Yes, I wish Open AI (maybe it is time for a rebrand) would be more open with what they’re doing. They released 4-o, it was great, then there was a huge outage and it got noticeably worse.
12 Likes
I work with prompts every single day for several months, I think I understand why you feel like the quality drops. In fact, it is always a big problem for GPT4 or GPT4o to solve complex problems in terms of instability and hallucinations. Every time I modified a prompt, I would usually run it 50 times to test whether it is stable. It is always unstable unless you are very very careful with what you say. I guess, the more you work with GPT, the more mistakes you will find that GPT makes. Maybe it is a sign that you are more familiar with GPT now.
7 Likes
Since day one, I had feeling they replaced GPT 4 with GPT 2, most of my prompts were worse than GPT 3, festered with mistakes, so I canceled my subscription. After all the hype for me it was kind of a big disappointment.
I do not know what is going behind the scene, but I hope problems will get solved.
3 Likes
AI Studio is doing a much better job lately… For free…
1 Like
GPT4o is a disappointment in terms of cognitive capabilities, always has been, especially when it comes to large context windows. People like it because it’s fast and writes a lot of code, but the quality is annoying.
2 Likes
Since launch day, I spent approximately 18 hours working with ChatGPT 4o, expecting it to be an improvement over its predecessor. Unfortunately, I found the quality of its code tasks, context awareness, and ability to follow instructions significantly declined. It became so unusable for coding that my entire team stopped using ChatGPT altogether. OpenAI seems to have reduced its capabilities. Now, I use Clause 3 Opus, and the difference is like night and day.
It is really sad to see them not do anything about it.
5 Likes
Sam Altman has spoken countless times about iterative development. He has also said that OpenAI has solid evidence that GPT-5 is smarter than GPT-4 and that GPT-6 will be smarter than GPT-5.
Considering that GPT-4 is a 2022 technology and OpenAI appears to already be working on version 6, “iterative development” for Altman means:
working toward AGI in the lab while customers iterate on different versions of GPT-4. 
1 Like
Altman also said GPT-4 was as dumb as it ever would be. A year of “GPT-4” branded models that only get dumber prove otherwise.
2 Likes
Desde el dia 17 de junio he esperimentado problemas con esta Chat GPT y todas sus competidoras ninguno te resuleve nada y da respuestas sesgadas o cortadas, y no veo en nigun lado que se mencione esto cambien de vpn pensando que era por la region donde me encontraba pero de igual manera no da respuestas, pense que era por usar el grauito y es lo mismo en la de pago, alguien mas esta experimentando esto?
Its funny, few weeks after each model release I notice these updates. I am hoping it is only the peak demand and not akin to battery draining moves attempted by the likes of apple in the past to generate more revenue.
I’ve noticed that the content of the replies to the gpt4 & gpt4o models under my account are all related to 3.5.
When I ask: “what’s your current model and knowledge cut off date?”, the response is always: “As of my last update, I am based on the GPT-4 My knowledge is current up until January 2022!”.
Obviously, January 2022 is the knowledge cut off date for GPT3.5.
As they say “never play on release day” seems to be true with AI also.
I Like this status page better then the official:
1 Like