Api image/text order with gpt-4v

Hi,

It’s quite possible that 4V and claude v have different architectures. It’s also possible that it’s a training issue. Also note that this is just a recommendation. It’s possible that this is what gave them the best results during internal evaluation.

with 4V you can have a bunch of images in your conversation, but performance can degrade if you have too many, or too many of them are too similar - interestingly, those are the same issues it faces when confronted with redundant text.

Overall, I think the answer to your questions might be, it’s hard to say.

I’d encourage you to evaluate different methods for your use case and figure out what’s best.

My go-to is to introduce the subject context, followed by the actual subject (whether text or image), and then the query/instruction on the subject. that seems to work great for almost all cases.

I’m thinking that working with LLMs is kinda like horse riding.

Is it a science? not really. Can you make a science out of it? sure.
Is it an art? not really. Can you make an art out of it? sure.

Are you a cowboy, or are you doing dressage? Or are you into horse breeding? :thinking: Which one’s right? Which one’s wrong?

I’d offer that a recommendation is just that - something to get started with. It’s quite possible that a third option might be discovered down the line that works even better, who knows. :confused: