I am using the openAI responses API to generate text responses with gpt-5, minimal reasoning and low verbosity. When generating a response, rarely the response output_text will include what seems like an intermediate reasoning step, something like: “Ok, now i will check the account. done. checked the account. The account has a certain account type. How do i handle this account type. Ok, i must follow instructions for that account type. Ok, done. Based on your account type, please […final response]”
It does return the final output text that should be sent, but it’s prepended by some reasoning.
How i’m parsing:
First, messages is defined by taking all elements of response[‘output’] where the output has type == ‘message’ and role == 'assistant'
And then, for each element of messages, and returning each element’s text value if the type == ‘output_text’. Then, I join them all together. I’m guessing it’s something to do with joining them all together, but i’ve got no idea how then i would differentiate between ‘final’ output text and intermediate output text.
I am on ZLR, and i’m finding this issue impossible to reproduce. Is this a prompt issue, or is there an issue with my parsing? It works correctly almost always, but rarely will return what seems like a non-final response above. I haven’t been able to capture a request that causes this, which is driving me nuts. I’ll add more logging on my end to see if i can capture one, but in the mean time, wondering if anyone else has run into this issue.
Hi,
We are facing the same issue, did you find a workaround for this ?
We’ve figured out that whenever it happens, the returned reasoning is empty. Thinking of adding a retry on empty reasoning.
I think you are using a poor AI model that starts with “gpt-5” that has no decent controls over the sampling done (to be even cheaper and save math?), and thus, cannot reliably signal “channels” in which to appropriately write its “commentary” and “analysis” with unseen reasoning, vs “final”.
The AI writes the final channel and proceeds to its post-trained thinking-style output anyway.
The only thing to be done is talk directly about what the AI is too poorly trained to know and do in your developer message, to reinforce where “unseen planning” goes, and try to improve the logits. You can read more about the “harmony” ChatML replacement that has trickles of information to inform your communication style.
Then escalate the appearance of authority of “developer”.
You’d have to write a “developer” book in order to cover every edge case of systemic bad model behavior, though.
The API will return two message types in the output array (in varying order, so we can’t just pick one) with content[0] populated - one will be the ‘real’ output and the other will be an intermediate reasoning step.
We are using the responses API statelessly in accordance with zero data retention. We have not seen the error again since enabling encrypted reasoning items (won’t let me post a link, but it’s in the docs) but I can’t say for sure that resolved it until we have a sufficient sample size. Will follow up if we still see it, and perhaps attempt another solution like the one you proposed above.
In a self-managed conversation (“store”:false), passing back the encrypted reasoning for placement may act a bit more like multi-shot in reinforcing to the AI “how to write” its responses in general. Good discovery.
To receive it along with summary in the “output”, with some booleans to turn it off in Python:
"include": [
*(
["reasoning.encrypted_content"]
if is_reasoning and include_encrypted_content
else []
),
# .. other include types not demonstrated
],
Also got this issue. In the output array, the reasoning text seems be marked as msg. so I can’t differentiate between them.
If the reason token is encrypted, how do we output the reason content?
curious how you guys want to output the reasoning text?