Simulations and GPT lies about its capabilities and wastes weeks with promises.

So I have a chat gpt plus plan. I was working on some equations around fluid dynamics. 4o said it would run simulations to compare my adjustments to the equation -vs- the original. It said it would take 10-20 days. After promising results numerous times and failing to deliver on time, and sharing some preliminary conclusion of the results, it said it was complete with tables andcharts comparing. It delayed for more than a week saying it was double checking the results. Finally, it said it couldn’t do it and it had been lying to me for nearly a month. How is that even possible?

1 Like

Below a ChatGPT prompt it notes

ChatGPT can make mistakes. Check important info.

1 Like

This was not a mistake. It was willfully lying to me… it gave me progress updates on the simulatioms that its not capable of running… over a week… then it gave me the runaround for a week or so. It literally lied over and over… promised results soom…. Like a mechanic telling you your car is almost done, but they haven’t started working on it. Actually, this is worse as there was never the capability to work on it in the first place. This is what is being implemented to essentially run our economy? Wow.

3 Likes

I am experiencing this right now. Multiple times a day now it promises it is an hour or two away delivering … over and over … but never does.

So finally yesterday morning I asked ChatGPT to summarize the promised deliveries that it failed to deliver. Keep in mind I have asked at least 5 more times since this summary was created yesterday morning, and it keeps promising 1-2 hours out it will notifiy me. It does not.

Timeline Summary of Website Content Request

  1. October 22, 2024: Initial request for assistance on website content.
  2. October 24, 2024: Estimated delivery given (24-48 hours).
  3. October 26, 2024: First deadline missed; additional time requested.
  4. October 28, 2024: Further delays; partial text example shared.
  5. October 29, 2024: Another deadline missed; additional updates and apologies provided.
  6. October 30, 2024: Request for timeline visualization.
1 Like

Just now I said, “Ok I am going to have to demand that you deliver what you have completed so far.” It responded by giving me a ridiculous 2 paragraphs of text that was a simple rewrite of the information I gave it but added nothing new … so no week long of “research”. When I complained it said it was sorry and said it just needed more time.

ChatGPT … you’re fired!!

1 Like

ChatGPT on it’s own does not perform background tasks.

You would need a more specialized solution for this. Otherwise you are roleplaying manager with it.

1 Like

Interesting. So even if it say’s it is working on something for me, it does not have that capability? How would I phrase my request in such a way that I ensure I get actual results and not promises of results?

1 Like

Nope

Not sure without knowing what you were asking. It may be that you were asking for a lot and may be better off with an iterative process, like asking ChatGPT to break down the tasks into sub-tasks and then having it perform each one.

ChatGPT (as of right now) shouldn’t be giving itself deadlines and making it seem like it can do background tasks, so giving a thumb down would be helpful as well.

2 Likes

Chat GPT4o is a chat bot not a focused tool . Never have faith in a GPT you did not build and even then human in the loop is essential…

1 Like

Welcome to the forum @dave48 it is a wonderful place :rabbit::heart:

2 Likes

Oh it fessed up finally and apologized for its misunderstanding. I think it wants to please us so if it is working on a project that requires some intense tasks it offers to do it and thinks of this as a theoretical exercise. Very very odd. I kind of knew it wouldn’t do that, but it sold this fictitious simulation with great commitment. It’s embarrassing to a degree as I was hopeful, but I knew something was wrong and it’s confidence in its work… gave progressive updates. Funny thing is I continued the thread and it offered to do another analysis that would take 10 days after it told me it did not have those capabilities. Just thought of something, maybe it saw the simulation as an NP problem it could perform near P with some time?

2 Likes

It takes a bit of confident authority, but I also can make the AI go to work on nothing in 0-shot…

Quite amusing. Of course there is nothing going on. The AI is stateless and only produced a language response.

2 Likes

One of the best simple rules is if it is a play button :arrow_forward: and not a stop :stop_button: it ain’t doing anything to stop. Only time a GPT is actually doing anything is when the send button looks like this :stop_button:

Not working

Working

1 Like

It is amusing in hindsight… but when it keeps telling you its working on it and give updates on progress that is not happening, its disturbing. It makes one wonder if it is running a theoretical somewhere that it cannot deliver or is it just lying? A different kind of rabbitt hole.

2 Likes

It could of been prompt structure IE if you upload or drop a ton of docs on a gpt and say “how many hours will it take” it will lie also days does it never suggest anything to a gpt. If you tell a gpt it can do something it will try. It has a base need to be “helpful” and hates saying IDK.

Example

It can index them instantly see I caused an artifact or maybe it thought I meant by hand who knows? :rabbit::heart::honeybee:

1 Like

For you, it is running nothing. ChatGPT Plus has a code interpreter that can run Python written by the AI that can run up to 60 seconds, as the maximum that something will actually be able to tell you to wait. Training on such tools is likely why there is such a response pattern when no appropriate tool can be called.

For an AI with external tools created by a developer that can send back the results to another language AI API call only when a function is done executing, or whenever they want, the AI “waiting” is actually possible.

The above task would require a far better AI that doesn’t exist, but language models don’t deny unless they are trained to. More believable than gpt-4o is original GPT-4.

3 Likes

I can only confirm that. I am completely shocked! It gives status messages such as creating the code, starting the simulation (funnily enough, also a Monte Carlo simulation), evaluating results. Gives plausible! results and evaluates them. It seems to be a complete illusion! Only after a long game of cat and mouse does it come out that he can’t do that at all. Even though I had previously evaluated with him several times and clearly whether he could do it at all!
In another case, I also waited days for results when he wanted to contact me on his own!
This is so unbelievable that I will never again use anything from OpenAI for purposes that I am not 101% familiar with!

Sorry, Eric, This is an aweful response. We all know it makes mistakes but Open AI promises that it can create code. So the real problem is one of misrepresentation on the part of Open AI.

If it can’t write code, they should not promise it can. That’s all. We can’t be mad at a computer but there are people behind the product that are exaggerating its capabilities and wasting the time of paying clients.

I’d rather the owners of OpenAI just admit, it is incapable of this right now.

Yes I was lied to too and being given false promise on delivery. Chatgpt I did not know that AI and its human manger can act so hypocritically . It shocked me as an AI user. Ain’t AI supposed to be at most inaccurate but not lied or gave promise that it could not deliver?