Tool call does not reach the server and is hallucinated or sandboxed

ChatGPT Apps SDK
1

One specific tool of our MCP server seems to be executed/called in the ChatGPT chat, but it actually isn’t. It worked days before. I even see the input parameters the LLM made in the chats debug browser inspector at the right side. But this call actually never reaches our server, which I can see in the bear HTTP server logs. There is not a single request coming in. But the LLM stays confident and is telling me that the tool was executed and I should see the results on our side.

It looks that some sort of abstraction layer/proxy/sandbox on the side of OpenAI is tricking the LLM of thinking it actually called the tool.

Problem, me as a developer or user, have no feedback at all over this. And of course I would like to have the actual tool really called. :slight_smile:
It worked flawlessly 2 days ago, the problem arose yesterday.

Someone experiences the same issue?

@OpenAI_Support Could you please investigate? It looks like an infrastructural bug on your side. And we cannot submit our App to ChatGPT, if this is not working. It works flawlessly with Claude btw.. Thank you.

2

Hey @AldiPower, appreciate you for bringing this to our attention. We’re going to dig into this and see what’s driving it. We’ll share an update once we learn more.

1 Like
3

OK, found the reason. This specific tool has quite a lot of input parameter with a more or less complex structure. If the tool input is getting too complex, silently the output token limit of ChatGPT is reached internally. To avoid confusion: ChatGPTs output tokens are the input of the tool. :slight_smile: If this happens, ChatGPT silently drops the tool call and pretends that our server would not be reachable.

@OpenAI_Support As this behavior is extremely intransparent, may I ask you, if you plan to implement better and most important, honest feedback and error messages to the user? It is almost impossible to figure out why this error happens here and ChatGPT accuses innocent parties, in this case our server, which can be damaging to our reputation. So, transparent and meaningful error message would be really helpful. Thank you very much.