MCP tool returns an image successfully, but ChatGPT shows only text - no image or widget

ChatGPT Apps SDK
, ,
1

Hi everyone,

I’m building a ChatGPT app that generates images. The tool completes successfully and returns the image, but ChatGPT often displays only a caption.

For example, ChatGPT replies “Here is your image” and describes it, but there is no image, no widget, and no visible error.

The MCP response includes the actual image bytes - not just a URL or a file reference:

{
  "content": [
    {
      "type": "image",
      "mimeType": "image/png",
      "data": "<base64-encoded PNG>"
    },
    {
      "type": "text",
      "text": "Here is your image."
    }
  ]
}

We also tried a minimal component, with the tool pointing to a registered UI resource:

{
  "_meta": {
    "ui": {
      "resourceUri": "ui://example/image-result-v2.html"
    },
    "openai/outputTemplate": "ui://example/image-result-v2.html"
  }
}

The resource uses text/html;profile=mcp-app and contains all its HTML and JavaScript.

After refreshing the developer connection and testing in fresh conversations on desktop Chrome:

  • A small PNG of around 3 KB displays successfully in the component.
  • A valid PNG of around 3 MiB produces only a text response.
  • In the failing case, no image element or component iframe appears.
  • Direct MCP checks confirm that the image is valid and the UI resource can be retrieved.

We fixed a CSP issue in our own component, which resolved the small-image case. The larger-image case still fails. We haven’t established an exact size threshold, so I’m not assuming this is definitely a size limit or entirely a ChatGPT issue.

Is there a supported, reliable way for an MCP app to return an image that ChatGPT visibly displays? Are there payload limits or additional requirements we might be missing?

Even a visible error would help. At the moment, ChatGPT says the image is ready while the user receives nothing.

I have a minimal reproduction using synthetic PNGs that I can share.

2

Did not check the exact details but reading your topic reminds me of this bug

The secret, just look below at the Related Topics

3

+1 same problem - the former thread doesn’t really offer valid workarounds - claude supports this as well

4

This is probably the closest match I’ve found to what I’m seeing: Apps SDK: Images uploaded return sediment://file_* IDs but model cannot access image content · Issue #170 · openai/openai-apps-sdk-examples · GitHub

It seems to describe essentially the same problem with uploadFile() and imageIds. Unfortunately, the issue has already been open for about nine months, so I’m guessing this is fairly low priority to fix.

I’ll keep trying to find a workaround and report back here if I get it working reliably.