How 'content' and 'structuredContent' in the tool response is handed to the model

[freetobe]

Hi everyone

I’m currently experimenting with building a ChatGPT app, and I’m a bit confused about how content and structuredContent are supposed to work when they are both returned from a tool.

According to the official documentation, it says:

“The widget reads those payloads through window.openai.toolOutput and window.openai.toolResponseMetadata, while the model only sees structuredContent / content.”

From this, I initially understood that the model would receive both content and structuredContent as part of its context.

However, in practice, when I create a tool that returns both content and structuredContent and then call it, it appears that only structuredContent is actually used by the model when generating responses (i.e., the ChatGPT-generated reply, not the widget UI). The information in content seems to be completely ignored by the model whenever structuredContent is present.

I’ve verified this with several tests using non-guessable values, and the behavior is very consistent.

So my questions are:

• Is this behavior intentional?

• Is content meant to be primarily for UI / widget rendering, while structuredContent is the canonical input for the model?

• Are there any best practices or tips on how to properly use content vs structuredContent together?

Any insights or clarifications would be greatly appreciated. Thanks in advance!