Which model is better for analysing an image?

API
, ,
1

I’m making an image analysis application and I can’t find information on the Internet which model is better able to handle image analysis, what do you recommend? choose between gpt-4o, gpt-4o-mini, o3-mini, o1-mini

2

I revised your message…

“I made an image analysis on my application comparing models because I couldn’t find information on the Internet. Here’s what I recommend for my particular use-case and desired output, which may be significantly different than model performance on yours.”

A small effort and you’d find the effect of:

[“o1-mini-2024-09-12 does not support attachments.”, “o3-mini does not support attachments.”]:

which is
{'error': {'message': 'Invalid content type. image_url is only supported by certain models.', 'type': 'invalid_request_error', 'param': 'messages.[3].content.[2].type', 'code': None}}

A little more investigation and the discovery is that gpt-4o-mini actually costs twice as much in dollars per image than gpt-4o at current pricing (and intentional multiplication of billed tokens).

Leaving one answer from your choices.

3

It’s strange, I’ve watched various reviews where the o1 mini and o3 mini models were used in image analysis. How did they do it then?

4 
--- Testing gpt-4o
The image is a random multicolored static pattern, similar to television static. It consists of numerous small squares in a wide variety of colors, creating a pixelated and noisy appearance.
CompletionUsage(completion_tokens=38, prompt_tokens=94, total_tokens=132, completion_tokens_details=CompletionTokensDetails(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0), prompt_tokens_details=PromptTokensDetails(audio_tokens=0, cached_tokens=0))

--- Testing o1-mini
Traceback (most recent call last):

openai.BadRequestError: Error code: 400 - {'error': {'message': 'Invalid content type. image_url is only supported by certain models.', 'type': 'invalid_request_error', 'param': 'messages.[0].content.[1].type', 'code': None}}

--- Testing o3-mini
Traceback (most recent call last):
openai.BadRequestError: Error code: 400 - {'error': {'message': 'Invalid content type. image_url is only supported by certain models.', 'type': 'invalid_request_error', 'param': 'messages.[0].content.[1].type', 'code': None}}

On the new Responses API, you are allowed to not read nor believe me and waste your money if you want:

--- Testing o3-mini
Assistant: {"response_to_user": "I don't seem to have received an image. Could you please try attaching the image again?"}