Recent decline in ChatGPT 4.0 responses to coding prompts

ChatGPT
,
1

I’ve been using ChatGPT 4.0 for a few months to help me write basic code, and it has performed quite well for my purposes. However, something changed about 1-2 weeks ago and it’s performance is much, much worse (details below).

Have any updates/features launched that might be causing this? I did notice the appearance of “memory updated” in some of my interactions and I’m wondering if this could be the cause. Should I clear my cache or turn it off?

Usage Details
I always begin a new prompt for code, and usually get everything I need withing 1-3 messages in total. My prompts are generally follow this format:

I’m writing a code in [language] and I’m using [package] to do [task]. My code currently does [task] well, but I want to modify it so that it can do [new task]. Specifically, I want the new code to [description of new task].

I’ve provided my current code below. Please modify my code so that it performs [new task] in addition to its prior functions.

Be sure to review my code before responding.

[my code]

This sort of prompt used to give me a general text response about how my code could be adapted as well as the updated code in a code block/window. However, the responses as of the past ~1-2 weeks or so are frequently incorrect or poorly tailored.

Example
Let’s pretend that “theme” is a general parameter of my code that has multiple options. In my prompt (formatted as described above), I ask ChatGPT that I would like to modify some aspect of “theme”. In my code, which I’ve provided to ChatGPT in my prompt, the theme is set to “theme=theme_C”.

Current Behavior (ChatGPT 4.0)
ChatGPT responds, but its response uses settings/parameters that apply only to the default theme setting (for instance “theme_A”). It completely ignores that fact that my code is using theme_C.

If I reply with: “You’ll notice that the code that I provided is using theme_C. Shouldn’t you response incorporate settings that are available to theme_C?”

ChatGPT apologizes and provides a better code that is suited for theme_C.

Prior Behavior (ChatGPT 4.0)
In most instances, ChatGPT 4.0 would provide code that was correct and tailored to my needs. Issues were almost exclusively caused by limitations of my prompts (omissions, poor descriptions, etc).

2 Likes
2

Hey mate, this sounds like an instance of ChatGPT becoming “lazy.”

Check out this discussion.

My suggestion, if you’re using ChatGPT Plus, is to make a CustomGPT that has your exact specifications.

2 Likes
3

Thanks. Looks like lots of other folks are experiencing the same problem.

It’s a massive thread, but I didn’t see much discussion about whether it could be related to the newish memory feature.

I’m curious if responses would improve if I:

  1. turned off the memory feature
  2. deleted old chats

I suppose I can try it out, but based on the size of that thread I imagine others already have.

I have no experience with CustomGPTs. Is it challenging to get one established? Realistically, I could dedicate no more than 1-2 hours trying to set that up. Is it feasible within that time frame?

4

Nah, I’d wager this has nothing to do with the new memory feature.

You’re right, they’re doing a ton of stuff in the background. But that particular feature will mitigate this type of thing. Can’t tell you how excited I am about it, honestly. :smiling_face_with_three_hearts:

You just have to train a custom model for yourself with your specific requirements. It might take all of ten minutes to make a basic formatting GPT. Just go make a new GPT, then converse with the GPT Builder. Give it code samples and anything else you want to document or output. Enable Code Interpreter.

If you have an hour it’s worth spending that time training your model with some back-and-forth. Tinker with it, refine your Instructions, upload reference material to the Knowledge Base, and how to properly reference your added files in the Instructions for quick access. It’s fun!

1 Like