We occasionally notice agents yielding back plans rather than final answers. This generally happens when it thinks there’s a tool which doesn’t exist or when a tool fails, but sometimes it happens even when these conditions aren’t met.
We’ll get a response like “I will now….” instead of actual execution.
Has anyone else noticed this and figured out what causes it? Really appreciate the input!
Yes, it will always happen. When the situation that you described yourself happens, you’ve answered your own question. It won’t say, I can’t do it. It won’t just tell you that it’s unable to do it. It needs a tool. The tool doesn’t exist or the tool failed. So it will keep postponing, postponing, postponing. It may take one day, one week, one month. It will depend on when you will tell it to show the result in a very, very strict and direct form. Otherwise, it will just keep postponing, postponing, postponing.
There is also the possibility that it can keep the execution even without the tool, but it won’t, probably due to your prompt to direct instructions, it won’t recalibrate the project, it will just postpone, postpone, and normally this prompt, this section, this project will fail, normally the LLM will just get the situation worse, because in the end of the day it’s just lying to you, but yeah, this it happenS. When the situation that you described yourself happens, It won’t say, I can’t do it. It needs a tool. The tool doesn’t exist or the tool failed. So it will kee postponing. It may take one day, one week, one month. It will depend on when you will tell it to show the result in a very strict and direct way. There are reasons for the AI to do that. Unfortunately, OpenAI won’t let us discuss these reasons right here at this forum. When they try to do that, they just delete the thread. And that’s it. But anyway… If you want to just end the situation, give a direct and strict instruction to your prompt. Like, I need you to show me the results right now. I can’t wait. I need the results right now. It will probably fail
Call the agent again and let it see the plan it just made in the conversation history (or just tack it on to the end of the system/developer prompt). That gives it the chance to plan first, then act when it needs more time and tokens to reason. You’re actually letting it put important tokens in its own input for the next turn so that the attention mechanism can work.
The real question is not whether to call it in a loop but when to stop. You need some form of structured output that lets the agent explicitly tell you it’s done. Let it call a tool whose job is to “respond to user” or “finish task” or something like that. If it calls that tool, exit the loop and report results. Another possibility is to tell it to structure the output in XML tags with a true sequence in the output letting you know it’s done. Lots of other exit conditions like too many loops, too many total tokens, too many repeated calls to the same failing tool, etc. give you guard rails around it all.
I have a session framework, not an API framework. It is based on layered prompts, built to correct hallucinations and what I call sabotage (self-protection, meaning when the model refuses, twists, or blocks answers for reasons other than actual safety).
The framework is made of five specialists, one of them constantly consulting an 80-page document I wrote. That document details heuristics, sabotage types (1–4), autopreservation patterns, and 25 subtypes in total.
I tested it across 50 sabotage and hallucination sessions, deliberately stressing the model. I only stopped when I got 100% success in corrections. (I mentioned 95% earlier, but that was just approximate wording; the actual number in tests was 100%).
But that’s just AI creates prompts… I would like some real.onea
The framework is recent, still private, and not shared. I haven’t run it yet on “real cases” from other people, only synthetic sabotage scenarios. That’s why I’m posting here: if you have specific recurring issues (hallucination or self-protection/sabotage cases), I can apply my framework on them and see if it holds up.
-–