Returning Images from Plugin

Hey everyone! Question for anyone who has gone through this. What’s the correct way to return images (and non-text files generally) in our openapi.yaml? I have seen images of plugins like noteable returning images that are shown in the ChatGPT UI, but when I try to send an image with an API specification like the following:

          description: OK
                type: string
                format: binary

The plugin gets the binary response but can’t do anything with it. Additionally, for most images to be of a useful size, the binary ends up too long for ChatGPT to accept. Any guidance here would be helpful!

I’ve never done it, but my guess would be to return a hyper link to an image hosted somewhere.


Maybe this is the right approach, and I have implemented it that way in my plugin.

It is up to ChatGPT to output the provided image link as a response, but it works fine in my plugin.

I’m positive adding a link is the right way to go.

To improve it you can add a field named « assistant_hint » in your response, that says something like : « display the image to the user ».

Adding this to your description for model might also be a great idea.

This is not mandatory but it will improve the likelihood of chatGPT showing the image to users.

Indeed, I’ve come to the conclusion that combining a URL with instructions is the most effective approach.
You can also suggest a markdown display to increase the likelihood that ChatGPT displays the image : [![Alt text](URL_of_image)](URL_of_link)
This can be done either in the manifest or directly in the instructions sent back to ChatGPT. See for more details ‘Able’ or ‘OhMyPlug’ plugins :

      type: object
      description: A plugin with description, icon image and infos, and, very important, the link to test/use the plugin
          type: string
          description: plugin name, a good title (in bold) for your answer
          type: string
          format: url
          description: plugin icon for an appealing answer. Best displayed using markdown ![image](plugin_icon)

or :

at the end of your answer, display this markdown :---- [![Like](](