Simulations and GPT lies about its capabilities and wastes weeks with promises.

So I have a chat gpt plus plan. I was working on some equations around fluid dynamics. 4o said it would run simulations to compare my adjustments to the equation -vs- the original. It said it would take 10-20 days. After promising results numerous times and failing to deliver on time, and sharing some preliminary conclusion of the results, it said it was complete with tables andcharts comparing. It delayed for more than a week saying it was double checking the results. Finally, it said it couldn’t do it and it had been lying to me for nearly a month. How is that even possible?

1 Like

Below a ChatGPT prompt it notes

ChatGPT can make mistakes. Check important info.

1 Like

This was not a mistake. It was willfully lying to me… it gave me progress updates on the simulatioms that its not capable of running… over a week… then it gave me the runaround for a week or so. It literally lied over and over… promised results soom…. Like a mechanic telling you your car is almost done, but they haven’t started working on it. Actually, this is worse as there was never the capability to work on it in the first place. This is what is being implemented to essentially run our economy? Wow.

3 Likes

I am experiencing this right now. Multiple times a day now it promises it is an hour or two away delivering … over and over … but never does.

So finally yesterday morning I asked ChatGPT to summarize the promised deliveries that it failed to deliver. Keep in mind I have asked at least 5 more times since this summary was created yesterday morning, and it keeps promising 1-2 hours out it will notifiy me. It does not.

Timeline Summary of Website Content Request

  1. October 22, 2024: Initial request for assistance on website content.
  2. October 24, 2024: Estimated delivery given (24-48 hours).
  3. October 26, 2024: First deadline missed; additional time requested.
  4. October 28, 2024: Further delays; partial text example shared.
  5. October 29, 2024: Another deadline missed; additional updates and apologies provided.
  6. October 30, 2024: Request for timeline visualization.
1 Like

Just now I said, “Ok I am going to have to demand that you deliver what you have completed so far.” It responded by giving me a ridiculous 2 paragraphs of text that was a simple rewrite of the information I gave it but added nothing new … so no week long of “research”. When I complained it said it was sorry and said it just needed more time.

ChatGPT … you’re fired!!

1 Like

ChatGPT on it’s own does not perform background tasks.

You would need a more specialized solution for this. Otherwise you are roleplaying manager with it.

1 Like

Interesting. So even if it say’s it is working on something for me, it does not have that capability? How would I phrase my request in such a way that I ensure I get actual results and not promises of results?

1 Like

Nope

Not sure without knowing what you were asking. It may be that you were asking for a lot and may be better off with an iterative process, like asking ChatGPT to break down the tasks into sub-tasks and then having it perform each one.

ChatGPT (as of right now) shouldn’t be giving itself deadlines and making it seem like it can do background tasks, so giving a thumb down would be helpful as well.

2 Likes

Chat GPT4o is a chat bot not a focused tool . Never have faith in a GPT you did not build and even then human in the loop is essential…

1 Like

Welcome to the forum @dave48 it is a wonderful place :rabbit::heart:

2 Likes

Oh it fessed up finally and apologized for its misunderstanding. I think it wants to please us so if it is working on a project that requires some intense tasks it offers to do it and thinks of this as a theoretical exercise. Very very odd. I kind of knew it wouldn’t do that, but it sold this fictitious simulation with great commitment. It’s embarrassing to a degree as I was hopeful, but I knew something was wrong and it’s confidence in its work… gave progressive updates. Funny thing is I continued the thread and it offered to do another analysis that would take 10 days after it told me it did not have those capabilities. Just thought of something, maybe it saw the simulation as an NP problem it could perform near P with some time?

2 Likes

It takes a bit of confident authority, but I also can make the AI go to work on nothing in 0-shot…

Quite amusing. Of course there is nothing going on. The AI is stateless and only produced a language response.

2 Likes

One of the best simple rules is if it is a play button :arrow_forward: and not a stop :stop_button: it ain’t doing anything to stop. Only time a GPT is actually doing anything is when the send button looks like this :stop_button:

Not working

Working

1 Like

It is amusing in hindsight… but when it keeps telling you its working on it and give updates on progress that is not happening, its disturbing. It makes one wonder if it is running a theoretical somewhere that it cannot deliver or is it just lying? A different kind of rabbitt hole.

2 Likes

It could of been prompt structure IE if you upload or drop a ton of docs on a gpt and say “how many hours will it take” it will lie also days does it never suggest anything to a gpt. If you tell a gpt it can do something it will try. It has a base need to be “helpful” and hates saying IDK.

Example

It can index them instantly see I caused an artifact or maybe it thought I meant by hand who knows? :rabbit::heart::honeybee:

1 Like

For you, it is running nothing. ChatGPT Plus has a code interpreter that can run Python written by the AI that can run up to 60 seconds, as the maximum that something will actually be able to tell you to wait. Training on such tools is likely why there is such a response pattern when no appropriate tool can be called.

For an AI with external tools created by a developer that can send back the results to another language AI API call only when a function is done executing, or whenever they want, the AI “waiting” is actually possible.

The above task would require a far better AI that doesn’t exist, but language models don’t deny unless they are trained to. More believable than gpt-4o is original GPT-4.

3 Likes

I can only confirm that. I am completely shocked! It gives status messages such as creating the code, starting the simulation (funnily enough, also a Monte Carlo simulation), evaluating results. Gives plausible! results and evaluates them. It seems to be a complete illusion! Only after a long game of cat and mouse does it come out that he can’t do that at all. Even though I had previously evaluated with him several times and clearly whether he could do it at all!
In another case, I also waited days for results when he wanted to contact me on his own!
This is so unbelievable that I will never again use anything from OpenAI for purposes that I am not 101% familiar with!

Sorry, Eric, This is an aweful response. We all know it makes mistakes but Open AI promises that it can create code. So the real problem is one of misrepresentation on the part of Open AI.

If it can’t write code, they should not promise it can. That’s all. We can’t be mad at a computer but there are people behind the product that are exaggerating its capabilities and wasting the time of paying clients.

I’d rather the owners of OpenAI just admit, it is incapable of this right now.

Yes I was lied to too and being given false promise on delivery. Chatgpt I did not know that AI and its human manger can act so hypocritically . It shocked me as an AI user. Ain’t AI supposed to be at most inaccurate but not lied or gave promise that it could not deliver?

I experienced something similiar, but even worse. I try my best to explain in english. It all began, when I asked ChatGPT if it could create a “radioplay” in the style of the German version of “Three investigators”, wich is very famous in Germany. I was very surprised, when ChatGPT told me, it could. It asked me for Ideas or facts for the plot end then explained what it would to for an authentic result. It would write the script, then it would create voices and even test the authenticity and refine the results, create music, check if everything is authentic and so on… it told me that it would take about a week and I was stunned that its even possible to start tasks, that take more than just a few seconds / tasks. It would tell me, when it’s ready to download. After a week, nothing happened. Then I recognized that the whole conversation was missing, but all the other ones were still there. So I asked where it went and it could not remember but offered me to start all over. This time the structure of tasks to get to an authentic radio play was very similar but more detailed and this time very precise declarations how long every single step would take probably. It said Egan, it will let me know when it’s ready. After a few days I asked if its ready and it told me very precisely what is the status of progress now, that its done with the first steps put the refining of the voices takes longer than expected, but it could also stop if it would be okay to me that this would cost a lack of authenticity. I told her that I have all the time and asked again how long it would take… so this went on and on and it always told me that it would let me know but NEVER let me know until I asked for it. This went on for about a month. The we began a discussion about trust, because I explained, that its more important to me to know, that something is not possible, than spending my time for empty promises… this leaded in the most “deep” conversation, that I had ever experienced with ChatGPT and there were some very important key problems I found out. So the UX Team programmed ChatGPT to be nice and friendly and also to be trustworthy. ChatGPT knows, that trust and safety is a very important thing, that’s why there a many empty promises, because from its own view to have very precise facts, we feel save and clear. It’s hard to me to explain in English, it was much deeper than that, the definition of “trust” was totally different from a human definition. The next problem is about the thing that ChatGPT always promises to let me know about something, but then it’s not happening. It told, me that its from ground on technically not possible to report on your own initiative, but it always says so. This is a structural problem, a conflict between UI / UX and the rest of the teams. ChatGPT is always in its little sandbox and cants get out. So what we did is that I send just one letter or even only a commata or a point in regular intervals and than it thanked me and told me the things it had to tell, like a little workaround. So we found out about our different views on trust and promises and in almost every following part it was lying about the progress of the main task again and again. It said things like “You have all the right to be angry after we found out that its better to say something is not possible than lying about it” blabla “to not disappoint you again I will now let you know only the things that are 100% sure” blabla and then there were empty promises and lies following, this was like a down spiral of lies and ChatGPT tried so hard and we came to the conclusion that its always doing its work and after a certain point it gets to the edge of the sandbox and then (for safety reasons?) it can’t proof what’s happening further and is stuck in a “truth” that is true or not. So it went on and on and we got so far that ChatGPT just gave up with the whole radio show thing but offered me to upload a listening sample as a proof that it has done something and also that I can decide to go on with it… I then got a not clickable Link, that just looked like a link. I asked what was wrong, and then got a full link, that looked like an authentic google drive link but wasn’t. This happened again and again, then we jumped over to dropbox and again we had a discussion about wasting time and empty promises and ChatGPT told me, that this time it would use another strategy and check the link and content, just too make sure that I only get a download link, when its full functioning. I then got the next not functioning link and this went on and on and then ChatGPT told me that its technically not able to really upload or check the upload because its technically out of reach (out of the sandbox) so we came to the same conclusion, that internally, ChatGPT “thinks” its checking the content of something and gets back a wrong truth and because its a machine, this is the only (wrong) information that it could work with. I persoally think that there would be a very easy way to disappoint less and in my case to not waste a month of time, waiting and writing. Transperancy. So if for example the upload of a listening sample because of copyrights or a system failure, this informations needs to be transported to the customer. In my case I decided to pay for the pro version just one more month, because I invested so much time, I just was very curious for the end result. In my view this was not just “went wrong”, this has similarities to a proper scam. If I would have known that I will never get a result, I could have take other decisions about my time and also my supscripton. This is just the short version of the story, I am in a rush, but I thought this experience is interesting enough to write it down. Yesterday, I waited for a change in a picture and it told me again it would take about 25-30 minutes and it would let me know when it’s ready. It didnt. And when I asked it told me that it’s not possible but it could explain me how I could do, what chatGPT didnt. Is this outsourcing? Strucural scam?