How I Solved the GPT Image Display Glitch and Should We Keep the Security Mechanism?

GPT builders
, , , ,
1

Recently, I encountered a tricky issue while developing a custom GPT that supports multiple characters. To maximize extensibility, I stored all character information, including avatar links, in a file. These image links, hosted on GitHub, were displayed in Markdown format.

For a month, everything worked fine, until suddenly, the avatar images stopped displaying. However, repeating the same content in the following chat made the images appear again. This issue seemed to trigger under specific conditions, unrelated to the image source, link format, or browser, and it wasn’t a network issue either.

I posted on the OpenAI forum (In custom GPT, some images in Markdown format cannot be displayed irregularly, but they can be reproduced stably under the same conditions) seeking help and found other users facing the same problem, but no solutions were provided.

After extensive testing, I discovered that images display normally if their link has appeared in a previous prompt. If the link only appeared in the file and not in any prior prompt, the images wouldn’t display. This appears to be a new security mechanism by OpenAI, which somewhat hinders creativity.

Should my assumption prove true, I urge OpenAI to reconsider this constraint. Links originating from code interpreters and files are inherently safer compared to actions. The rationale behind this security protocol is unclear to me. To monitor whether this bug or security feature will persist, I’ve specifically created a GPT to test this. https://chat.openai.com/g/g-horVE39hs-does-custom-gpt-support-md-image-only-in-file

I warmly welcome everyone to share your experiences or opinions.

2 Likes
2

Hey folks, thanks for flagging this. We have resolved a few issues around generation URL’s. There are a few known edge cases that don’t work well:

  1. Having the model construct full URL’s on the fly (you should instead pass the full URL back in an Action response)
  2. Having the model fill in URL details (same as above, you should pass in the full URL and all UTM parameters)
1 Like
3

@logankilpatrick Can you link to a sample action where this works, and the prompt to convince the chatgpt to display an image using the returned link.

What I see is that the image link looks good, but the chatgpt browser-side client removes the image from the output after the llm tries to write it out. The entire URL and markdown image tag are removed.

1 Like
4

I pass the full URL, but the markdown and the URL are removed right after they are displayed (no image is rendered).

5

I have the same issue. I created a Custom GPT to answer questions about an intranet app at work. (It generates a PDF runsheet for agency care workers.) I uploaded code for the application. When asked what does the app do, the custom GPT will read the code (Django templates, views) and tell me what it does. When asked to generate an images DALL-E goes kinda nuts. My app is a very prosaic intranet app for aged care. Plain Bootstrap 4 HTML. Nothing fancy. So I uploaded four images (screenshots of the app) and told the Custom GPT to answer questions about how it looks using the images. It looks like it writes something in Markdown/URL but the text disappears almost as soon as it appears on screen and you cannot see the image.