OpenAI has deprecated direct image processing through its API

OpenAI has deprecated direct image processing through its API - what are alternatives?
Need help … related
I have a code wich captures images was sending it to OpenAI via API request to identify the object in the imge.
For example image contained a Tre Stelle - Feta Cheese container - the request was to identify the Brand, Category and Type of the object.
Prior I was getting Bbrand: Tre Stelle, Category: Dairy, Type: Yogourt …
I setup a Google Vision account and API … I send the image up to Google but I am getting a lot of data but not what I need … Google will find and get the text but is going to be everything what is written on the containre … so its just reading the container. I f I ask for label recognition … kind of the same thing … will not identify the container it self …
OpenAI model was working like a charm … Why did they deprecated … anybody would know?

Could anybody give some advise?

2 Likes

Are you confusing “deprecated” for “not working correctly for you?”

Or have you confused a past model being shut off with those that continue to offer computer vision, such as gpt-4o or gpt-4-turbo?

2 Likes

Nope … :slight_smile: please find under what I found on the OpenAI related:

" OpenAI has deprecated direct image processing through its API, so if you’re using openai.ChatCompletion or other similar methods for image-related tasks, they will no longer work in openai>=1.0.0"

I am talking about direct vision image processing via API … if you upload an image to the Chat bot then yes it is working.
When you upload an image via API request is not workiing anymore.

If I am wrong and you found a way to upload an image via API request to then pls share … :slight_smile:

Even if you have an image on your system with any container food related or not pls try to upload it and ask the response to be Brand:, Category: and Type : … let AI to identify the object.

This is a snipet of the code related to the image upload:

def encode_image_to_base64(image_path):
with open(image_path, “rb”) as image_file:
return base64.b64encode(image_file.read()).decode(‘utf-8’)

def send_images_to_ai(encoded_images, item_code):
# Prepare messages list with only available images
messages = [
{“type”: “text”,
“text”: (
"Identify the object and categorize it specifically under the most applicable Category within "
"Dairy, Non Dairy, Fats, Meats, Grain Product, Meat Alternative, Non Alcoholic Beverage, Spirits & Liquors, Dietary Supplement, Condiment. "
“Provide:\nBrand:\nProduct Type:\nCategory:\n\nNothing else.”
)}
]

# Add images conditionally based on availability
if 'esp32_1' in encoded_images:
    messages.append({"type": "image", "image": encoded_images['esp32_1']})
if 'esp32_2' in encoded_images:
    messages.append({"type": "image", "image": encoded_images['esp32_2']})
if 'esp32_3' in encoded_images:
    messages.append({"type": "image", "image": encoded_images['esp32_3']})

payload = {
    "model": "gpt-4-turbo-2024-04-09",

“model”: “gpt-4-vision-preview”, was working perfectly

    "messages": [
        {
            "role": "user",
            "content": messages
        }
    ],
    "max_tokens": 50
}
      
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {api_key}"
}
response = requests.post("https://api.openai.com/v1/chat/completions", headers=headers, json=payload)
if response.status_code == 200:
    response_data = response.json()
    print(json.dumps(response_data, indent=4))  # For debugging, remove or adjust as needed
    return response_data  # Return the response data for further processing
else:
    print(f"Error in image analysis API request: {response.status_code}")
    logging.error(f"Error in image analysis API request: {response.status_code}")
    return None

The commented model was working till Nov 18th after nothing related works.

They haven’t deprecated image processing through the API. That particular model was an early preview model that has been deprecated. Most new models can perform inference on images now, for example gpt-4o and gpt-4o-mini.

In general any *-preview model will be phased out in favour of a general availability (GA) model.

3 Likes

None works for me … if you have any advise please let me know.
What model shuodl I use then?

I replaced the Model ith the gpt-4o and gpt-4o-mini and I get the same error

Analyzing images with AI…
Error in image analysis API request: 400
ERROR:root:Error in image analysis API request: 400
Failed to get a valid response from AI.
AI analysis did not yield usable results.
AI analysis did not meet criteria for further action.

The API key is good and working tested it with a simple code to check the API key connection and functionality.

Nope … :slight_smile: please find under what I found on the OpenAI related:

" OpenAI has deprecated direct image processing through its API, so if you’re using openai.ChatCompletion or other similar methods for image-related tasks, they will no longer work in openai>=1.0.0"

I am talking about direct vision image processing via API … if you upload an image to the Chat bot then yes it is working.
When you upload an image via API request is not workiing anymore.

If I am wrong and you found a way to upload an image via API request to then pls share … :slight_smile:

Even if you have an image on your system with any container food related or not pls try to upload it and ask the response to be Brand:, Category: and Type : … let AI to identify the object.

This is a snipet of the code related to the image upload:

def encode_image_to_base64(image_path):
with open(image_path, “rb”) as image_file:
return base64.b64encode(image_file.read()).decode(‘utf-8’)

def send_images_to_ai(encoded_images, item_code):

Prepare messages list with only available images

messages = [
{“type”: “text”,
“text”: (
"Identify the object and categorize it specifically under the most applicable Category within "
"Dairy, Non Dairy, Fats, Meats, Grain Product, Meat Alternative, Non Alcoholic Beverage, Spirits & Liquors, Dietary Supplement, Condiment. "
“Provide:\nBrand:\nProduct Type:\nCategory:\n\nNothing else.”
)}
]

# Add images conditionally based on availability
if 'esp32_1' in encoded_images:
    messages.append({"type": "image", "image": encoded_images['esp32_1']})
if 'esp32_2' in encoded_images:
    messages.append({"type": "image", "image": encoded_images['esp32_2']})
if 'esp32_3' in encoded_images:
    messages.append({"type": "image", "image": encoded_images['esp32_3']})

payload = {
    "model": "gpt-4-turbo-2024-04-09",

“model”: “gpt-4-vision-preview”, was working perfectly

    "messages": [
        {
            "role": "user",
            "content": messages
        }
    ],
    "max_tokens": 50
}
      
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {api_key}"
}
response = requests.post("https://api.openai.com/v1/chat/completions", headers=headers, json=payload)
if response.status_code == 200:
    response_data = response.json()
    print(json.dumps(response_data, indent=4))  # For debugging, remove or adjust as needed
    return response_data  # Return the response data for further processing
else:
    print(f"Error in image analysis API request: {response.status_code}")
    logging.error(f"Error in image analysis API request: {response.status_code}")
    return None

The commented model was working till Nov 18th after nothing related works.

Sorry, image uploads were broken for a little bit today. (You can keep track of issues like this on our status page: OpenAI Status) gpt-4-vision-preview was deprecated last month, but you can use our latest models like o1, GPT-4o, or 4o mini. Can you try again?

4 Likes

This is what is not working for you:

Because you are using an obsolete format.

If you want to send a base64 image as part of a user role message, the data will need to be formatted correctly for the API SDK >=1.0 and for chat completions on modern API models.

messages.append(
    {
        "type": "image_url",
        "image_url": {
            "url": f"data:image/png;base64,{encoded_images['esp32_1']}",
            "detail": image_quality,  # "high" or "low", defaults to "high"
        },
    }
)

It says "image_url" but I assure you this is the correct usage. Ensure you change the MIME type to match file type discovered by an image library if you want ultimate compatibility.

Since a chat API request may have many messages of different roles, I would rename this variable in your code from messages to something like user_content. (Within the user multimodal content list are “items” or “blocks”, of either type:text or type:image_url, not messages.)

2 Likes

Thank you for the tip … worked I can upload image but the analisys is crap.

Analyzing images with AI…
Payload size: 103301 bytes
Raw AI Response:
{
“id”: “chatcmpl-AskCQcY4RQoGTNkg83Q8rdSipXZHW”,
“object”: “chat.completion”,
“created”: 1737610666,
“model”: “gpt-4o-2024-08-06”,
“choices”: [
{
“index”: 0,
“message”: {
“role”: “assistant”,
“content”: “Based on the analysis of the images, here are the details of the identified object:\n\n- Brand: Smirnoff\n- Product Type: Vodka\n- Category: Spirits & Liquors”,
“refusal”: null
},
“logprobs”: null,
“finish_reason”: “stop”
}
],
“usage”: {
“prompt_tokens”: 69807,
“completion_tokens”: 40,
“total_tokens”: 69847,
“prompt_tokens_details”: {
“cached_tokens”: 0,
“audio_tokens”: 0
},
“completion_tokens_details”: {
“reasoning_tokens”: 0,
“audio_tokens”: 0,
“accepted_prediction_tokens”: 0,
“rejected_prediction_tokens”: 0
}
},
“service_tier”: “default”,
“system_fingerprint”: “fp_4691090a87”
}
Identified Text: Based on the analysis of the images, here are the details of the identified object:

  • Brand: Smirnoff
  • Product Type: Vodka
  • Category: Spirits & Liquors

The image uploaded is a Allen’s Fruit Punch Juice
Result should be Brand: Allen’s, Type: Fruit Punch, Category: Juice

I do not know from where gets the Smirnoff, Vodka and spirits & Liquors … :slight_smile:
In the previous model was so accurate …
The whole logic is Motion detection event → camera captures image → resize image → encode image → sned to AI for recognition → response → parse response in to the final result.
Checked the image and it is cristal clear showing the Allen’s fruit Puch carton …
Any sugestions?

You have your choice of using high detail or low in that message format. Low is resized down to 512px maximum, so small text may be hard to make out. High detail has the imaged tiled by the server so there may be somewhat confused understanding.

You can still use gpt-4-turbo. You also have three different version models of gpt-4o, where gpt-4o-2024-05-03 is the first released, at higher price.

Figure that the amount charged per token also relates to the qualities of applied computation.

You will have higher quality if you instruct the AI to write a full description and analysis of what it sees, and only after that proceed to “object” extraction of the main object.

Then understand that the AI isn’t magic. You are likely the first to show it a jug of a brand’s fruit punch juice.


OpenAI recently distracted vision model attention with a whole bunch of unseen wasteful prompt text and prohibition that steals context focus. So there’s that.

1 Like

The previous model prior to Nov 18 I was able to upload all sort of food containers, cans, jars … , bevarage alcoholic and non alcoholic bottles.
The identification was spot on no issues.
Till you advised related to the upload condition was not able to upload any images.
Since I am able to upload under the exact same conditions the images is really bad the identification.

However if I use the Caht from my phone or computer upload the identical image and ask the exact same question the response is till spot on … just the API request is not performing at all.

Tried the gpt-4o-05-03, and the otehr models still I am getting the identical wrong answer.
I also instructed to have a great detail of the image as mentioned … the problem is that the initial identification of the object is completely wrong.
Like instead to Allens Fruit juice … I get Smirnoff Vodka … :slight_smile:
What I am trying to say is that the initial object is wrong.

The project was working perfectly initially…
Will try to fine tune the initial request … maybe that need to be changed also … but my conundrum is why if I use the Chat is perfectly correct …

IF this helps - the list of models are all here:
https://openai.com/api/pricing/