I have been developing a ChatGPT app recently, and I keep encountering weird behaviors while testing my app.
First of all, it seems like sometimes, for large JSON payloads exceeding some seemingly random size, the output of the tool call gets truncated, making the model unable to extract the data it needs from the tool’s output.
More recently, ChatGPT started behaving weirdly in the way it called my tools. When explicitly asking it to pass certain parameters to my tools, it would tell me it was doing so, but I would not see the right payload being sent through to my app—or I would even see totally wrong payloads. It really seems like there’s some sort of other runtime changing what is being sent to my app instead of having the main agent thread do it.
Would anyone have any sources explaining why these behaviors are happening?
Thanks!
Hey @pldesch
Thanks for flagging this, and sorry for the late reply. Unexpected arguments and incomplete results can make app testing confusing.
A couple of checks that may help:
- Try a smaller response with the same prompt. The tool-result reference explains that
structuredContent and content are visible to the model, while tool-result _meta is only available to the UI component. Keep any data the model needs in the model-visible fields. - Review your parameter descriptions, including examples and allowed values, then compare the expected arguments with those received by your server. The tool metadata guide covers this testing approach.
- Use MCP Inspector to test your tools directly with the intended arguments, then compare those inputs and responses with the actual tool calls and results observed in ChatGPT.
If this is still happening, please add a redacted minimal reproduction and a recent timestamp with your time zone to your existing support request. If you don’t have one, email support@openai.com or contact OpenAI Support. Remove credentials and private data before sharing.
~ Ankit