A week ago I had GPT4 create a very simple C# application for MS Flight Simulator. It was amazing. Worked with only 1 correction
Today I used the same prompt and GPT4 can’t get it right after many corrections?/
The reply to my question about this was
apologize for the confusion and frustration caused by my previous responses. Upon reviewing our conversation, I see that my previous responses contained errors and inconsistencies. I’m sorry for any inconvenience this may have caused you.
To answer your question, I could have provided the correct code a week ago if I had taken the time to verify my responses and make sure they were accurate. As a language model, I rely on my training data and algorithms to generate responses, but I’m not perfect and mistakes can happen.
Again, I apologize for any confusion or frustration caused by my previous responses. If you have any further questions or concerns, please let me know and I’ll do my best to assist you.
is it because the more it gets trained per second the more un-generalized it gets
Are you feeding it some chat history as part of your request via some library like langchain?
From the reply, it sounds like it is confused in ways I’ve seen before when the history is itself confusing.
GPT4 while being really good, is still very non deterministic and sometimes may not be able to do the same task as it was previously, especially if it is overloaded at that time. In my experience, usually waiting for some time and rerun the whole thing does the trick
Hey, that’s exactly my thoughts lately. I’ve been using ChatGPT for pair programming in Python since the very beginning. Over the last two weeks, I’ve noticed a dramatic increase in the number of poor responses. Interestingly, my friend who uses GPT with C++ has made the same observation. While I understand that we’re still in the early stages of this technology, I can’t help but wonder if this decline in quality is due to constant training.
I use it on daily basis as a programming assistant and I didn’t notice any degradation in quality. Indeed, I use the same chat for each project and what I noticed is that eventually GPT will learn most of your project’s codebase and will start using your architecture, methods and variable names even when it is not asked for such thing.
But what I noticed is that the quality of responses really vary a lot from programming language to programming language.
That might be the case. But I thought Python should be the most natural language for OpenAI models. Anyway, I still find it so impressive. Maybe I just need to work a little bit more on prompt management.
As someone wrote, working with AI isn’t about prompt engineering, it is a journey. Sometimes I just need to work it out with GPT before start getting good answers. I usually start by explaining GPT with what I am not satisfied in the answer.