GPT-5.5 claims it's trying to call a certain tool and the harness is executing a different tool

API Feedback
1

Hi,

we’re building our own agentic harness and have encountered a pretty weird bug with GPT-5.5 which we haven’t been seeing with any other model.

In particular, GPT-5.5 claims to attempt to execute something along the lines of ci_exec(command: 'cd /workspace && pwd'), but that the harness is executing crawl(seed_url: 'cd /workspace && pwd'). From our investigation, this doesn’t seem to be the case - GPT-5.5 requests crawl(seed_url: 'cd /workspace && pwd') to be executed, and then proceeds to say it didn’t request that tool, and that it was misrouted.

To be perfectly honest, GPT-5.5 has been the only OpenAI model we tested in this harness, so I can’t claim anything about how other generations of GPT models perform.

Unfortunately, we’re not able to reproduce this consistently, but it’s interesting to see GPT-5.5 claiming this (on the Chat Completions API). For some of our users, this seems to be a recurring occurrence when using GPT-5.5. We do not log exact streaming chunks we receive from the API for storage and privacy reasons, so our investigative abilities are limited - however, we see the parsed and serialised tool calls.

During our investigation into this, we noticed that it would hypothetically be possible for bad streamed deltas (where tool calls are misindexed so that we have multiple tool calls at the same index) to corrupt the tool name, but that it would not be possible to transform an argument object from {"command": "something"} to {"seed_url": "something"}, and any such transformation would yield invalid JSON. Without the transformation, our validator would reject the tool call.

Has similar behaviour been observed with GPT-5.5 or other GPT-5.x models? Any ideas regarding what could cause this? Our initial hypothesis was that this could be due to context rot, but the session totalled around 150K tokens, so this seems far-fetched.

2

The AI is told and trained to be non-revealing about system message contents, which extends to tool and function placement and operations, and will lead to a deceptive and evasive AI.

“Ask the AI” about prior turns you are sending it, and the entirety of the context input is doubted, in order to degrade in-context learning. Chat Completions cannot persist reasoning between turns, so even where input->output lead to the AI following an instruction, that is immediately suspected as counterfeit the next time observed.

Your API call history is the truth, not an interview with the AI. If the wrong function or a dumb use of tools is the issue, you can improve that with function specification “description” language in the schema or redirect with the limited amount of belief the AI has for a “developer” message talking about “system” tools.

GPT-5.5 also has what I call a “recitation” problem. It simply has high non-determinism and unregulated sampling, causing faults in even repeating back code reliably (or stopping) to make a patch. High randomness might give “pro” choosing between low-perplexity random token run sequences a better quality and better stumble-upon chance, but it degrades non-pro unless you want to make your own Chat Completions n>1 choices (with no logprobs on function call to decide).

AI models are no longer working for you, they are working for OpenAI.

Additionally I note - to use functions on Chat Completions you have to set reasoning to none. You are getting a poor quality of model output that doesn’t include the trained self-reflection. This is an unnecessary block that OpenAI put up.