I am not an advanced user, just a regular user using chatgpt for a few months. I have created several GPTs to help me in different topics (coding, analysing files,…)
I might not have written very advanced prompts, however I did not change them since a few month so I would expect to have at least the same quality of results over the time.
It’s not the case, my agents are becoming more and more useless, taking me a lot of time to get the result I expected and I finish often by taking more time with the agent rather than doing it by myseff
Among the regression, I have noticed
-
Answer precision : my agents is ommiting crucial information. For example, I ask often the agent to analyse and optmise my pieces of code. Before, it was working perfectly, now they are lines of code that are missing. I tell the agent it forgot some crucial code and the agent is saying sorry, bla, bla bla and then propose a new code but this time omitting some other information. Even with short lines of code (less that 100 lines)
-
Fake Technical problem : I ask, when I notice that the agent is proposing wrong code to first look at some documentation (that i have provided in the agent’s intructions) and the agent is answering it has a technical problem for browsing the documentation online. When I ask what is the problem, the agent is telling me that it doesn’t have the capability to browse websites. I answer, “you can, I gave you the ability” and it answers, “yes, sorry for the error” and it browse the website. I did not have the problem a few month ago.
-
code exhaustivity : I can bearly never have the full code at first time, the agent is always trying to replace pieces of code with comment even if i asked explicitely not to do so. I did not have the same problem with the same agent a few month ago.
-
retrieval errors : i have noticed many times that not only the agent cannot retrieve information on xlsx or cv but also can mix data ! for example, I was giving some performance test result on a website (5 octoperf report) and the agent totally messed the data. A few month ago, the same reports with the same agents were analysed perfectly.
-
totally useless knowledge base : the way to get the agent accessing the data in KB is an enigma for me. For example, I provide some xls files (well structured and cleaned) that have some data on applications. Most of the times when I ask an information about an application, it tells me it couldn’t find any information. When i say, you can find this information on this document, this time I may have a change to get the information. This have been since the beginning, I don’t use kb anymore, i drop files on the conversation
-
Error generating: i’m getting the “error trying to generate…” very often now
So is it just my imagination ? is it happening only to me ? Is there an explanation for that even if I don’t provide factual figures ?