Chat with images is rolling out now

Actually a good idea, it would be quiet interesting to see what happens :thinking:

1 Like

After asking, someone on Discord fed it a page, and it identified it, but they didn’t try to translate it …

2 Likes

Alright, it was a long shot :laughing:

Just got done reading the research paper, and I’m really impressed so far, it’s much better than I expected.

1 Like

After giving it a shot I can confirm that the results are not spectacular.
It mostly goes on and on about the document being medieval, takes a stab at the style (Gothic, 12-15th century) but never really makes any interesting statements.

PS. Sharing links with images is not yet supported. @PaulBellow

1 Like

No worries. Thanks for taking a stab at it! Was curious what it would “guess”… there’s been a lot of theories over the years.

ETA: Saw another screenshot on Discord but it wouldn’t guess… It seems like it’s relying on textual stuff rather than the image… or taking what it “knows” about the image “it’s the Voynich” document but is just gathering vectorized data about the “image”? hrm…

1 Like

It does make one wonder exactly how the prompting and context of images works in GPT-4. Can it be trained by example images? Does it have the context required to hold image data or is this processed by a different sub-model of the architecture that only returns language?

Thought it would be rolled out together with Dalle3 until I failed to find it in my chatgpt UI. No?

Yeah perhaps one needs to prod GPT-4 using some prompts and not just ask it to describe what is the document. Ask it to find patterns or whatever and maybe it can tell us more. I think like for us human, you show one a picture of Mona Lisa and the person will just tell you it is Mona Lisa. But if you tell the person about the smile, scenery, etc. and perhaps the person might give their interpretation.

1 Like

It’s something worth trying and if it locates Wally, I will be impressed.

2 Likes

Bonus question: ask for the precise bounding box, then feed the results and the image to the advanced data analysis tool and ask it to draw it :cowboy_hat_face:

2 Likes

I wonder if there’s any way to generate an attention heatmap of the image and, if so, how well that would correlate to locating Waldo?

4 Likes

Still waiting here. Anything else cool you’ve tried? Give it some history stuff?

2 Likes

I’m also waiting, that was appropriated from twitter.

2 Likes

Ah, thought it looked familiar!

Hope your weekend is going okay.

1 Like

Yup, all good. Just watched Sam Altman chatting with Joe Rogan, was a fun interview.

Another,


1 Like

I’ll trade my vision for your painting :smiling_face_with_tear:.

Here’s to hoping some powerful API capabilities. Bounding boxes and labels would be sweet as well

1 Like

Still haven’t received access. Is it only on the phone app? (I checked both)

There is currently a limited number of beta testers with access, this will be increased in time, please be patient while testing is being done.