Options for presenting user with non-text information?

I have my proof-of-concept plugin working and I just have to say how MUCH EASIER it was than my experience trying to do the same thing with Alexa. The natural language abilities of ChatGPT provide such a great user experience and magically handle context and setting the plugin’s API parameters – simply AMAZING.

My plugin could provide more value if it were able to present information as images and/or tables. What are the supported formats for the information returned to ChatGPT and how do I know what ChatGPT will do with it? The documentation says:

  • The model might include links returned from API calls in its response. These will be displayed as rich previews (using the OpenGraph protocol, where we pull the site_name, title, description, image, and url fields)"

  • The model can also output markdown (like an image URL from your API) and the ChatGPT UI will render the markdown in the UI.

I’m going to play with these to see what works, but, for instance, if I return a PNG image, will ChatGPT show it in the conversation? What about an HTML table or a video?

Finally, is there any way to have ChatGPT download a file to the user’s computer, or display a link that would do that? I realize there are important security considerations with this, but it’s a capability that would provide a lot value to users of my plugin.

If you return code (either markdown or HTML) to embed an image, it should render in the user’s browser without issue.