The previous Gpt4-turbo model supported image urls being part of a tool response which was useful for tools that responded back images to the models as a result of the tool call.
The new gpt4o models have apparently lost this functionality and can only receive images from messages with user role.
Is there a way to workaround this regression or is there any plan to fix the issue?
Example request:
[
{
"role": "user",
"content": [
{
"type": "text",
"text": "Navigate to google.com website."
}
]
},
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_MsDL8L4dUgJwxW4UKMW9XK6c",
"type": "function",
"function": {
"name": "navigate-to-website",
"arguments": "{\"url\":\"https://www.google.com\",\"keywords\":[\"search\",\"Google\"],\"searchDescription\":\"Access the Google homepage to perform web searches and explore Google's services.\"}"
}
}
]
},
{
"role": "tool",
"tool_call_id": "call_MsDL8L4dUgJwxW4UKMW9XK6c",
"content": [
{
"type": "text",
"text": "You have navigated to the website."
},
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoAA...==",
"detail": "high"
}
}
]
}
]
API Response:
{
"error": {
"message": "Invalid 'messages[3]'. Image URLs are only allowed for messages with role 'user', but this message with role 'tool' contains an image URL.",
"type": "invalid_request_error",
"param": "messages[3]",
"code": "invalid_value"
}
}