I’m making a prototype of a plugin that is turning out to be really cool, but it needs to print images, I didn’t find anything in the documentation that shows how to do that… can someone give me a practical example in python?
I tried use markdown tags, tried to pass the url of the image directly, I don’t know what to do, the image simply doesn’t print and the documentation doesn’t show how to do it.
The documentation just mentions:
- The model might include links returned from API calls in its response. These will be displayed as rich previews (using the OpenGraph protocol, where we pull the site_name, title, description, image, and url fields)"
- The model can also output markdown (like an image URL from your API) and the ChatGPT UI will render the markdown in the UI.
4 Likes
The documentations said that you need to converse with the AI naturally in your endpoint description to let it know what it should do and how it should display the images and respond to those requests, it should work:
Thanks, it worked, I told the AI to print the image of the markdown code and it worked, hope this helps someone who is experiencing the same problem!
“Show this markdown image: here the markdown code do show image”
2 Likes
Glad it worked, there is a great guide written by @wfhbrian that should triple the relevance and performance of your plugin if done right
2 Likes
I wrote a bit about rendering images and markdown here. In my experience just a bit of prompting helps:
1 Like
Very good guide and your overall blog has so many goodies
Thanks for sharing
1 Like
I’m testing the capabilities of ChatGPT. I’m collecting the results in Github below. I did verify that by giving the URL in JSON format works with png, webp and gif.
{
"imageUrl": "http://localhost:5003/assets/195.png"
}
2 Likes
in my case, I get the images via API, and most of the times the URL of the image doesn’t have the classic .png/.jpg extension, and chatGPT doesn’t recognize it.
i tricked it by simply appending .png to the end of the image url anyway!
I have another question that MAYBE has no solution, at least at the current moment of plugins, maybe someone has a hack.
Is there any way to show a loading or something and then just print the image? This thing of showing the image forming from a URL is very ugly.
When we print an image, ChatGPT is typing the URL letter by letter… then we see something like:
1 Like
Just add the content-type to the header. It will work without a file extension
I’ve tried various prompts trying to suppress this and it still shows the URL. A URL shortener would at least speed things up, making it less painful.
I have no control of the header, is a public database of images
I am trying to reproduce the Astrodaily plugin but no image is showed in chatGPT
I mounted the folders where the images are stored, the image is found but nothing is showed
app = FastAPI(dependencies=[Depends(validate_token)])
app.mount("/.well-known/", StaticFiles(directory=".well-known"), name="static")
# Mount "img" subdirectory at path "/img"
app.mount("/img/", StaticFiles(directory="img"), name="img")
...
@app.get("/nasa", response_model=NasaResponse)
async def nasa(dependencies=[Depends(validate_token)]):
try:
# Log the list of files
logger.info(f"Nasa before Current working directory: {os.getcwd()}")
files = os.listdir('.')
logger.info(f"Files in current directory: {files}")
imageURL="./img/nasa_hubble.jpg"
try:
# Get the file size
file_size = os.stat(imageURL).st_size
# Log the file size
logger.info(f'The size of the image file is {file_size} bytes')
except FileNotFoundError:
logger.error(f'The file {imageURL} was not found.')
except PermissionError:
logger.error(f'Permission denied for accessing the file {imageURL}.')
return NasaResponse(imageURL=imageURL)
except Exception as e:
logger.error(e)
raise HTTPException(status_code=500, detail="Internal Service Error")
What am I missing?
You have to give it to GPT in markdown Notation and then he’s Got to want to cooperate lol.
Took a lot of patience but if works in web requests
