It doesnt seem like GPT-4o can take in assistant generated images in the conversation history. When I move from gpt-4-turbo to gpt-4o and pass in an image_url generated by the assistant in the messages history, I get this error:
OpenAI Exception: Error code: 400 - {‘error’: {‘message’: “Invalid ‘messages[10]’. Image URLs are only allowed for messages with role ‘user’, but this message with role ‘assistant’ contains an image URL.”, ‘type’: ‘invalid_request_error’, ‘param’: ‘messages[10]’, ‘code’: ‘invalid_value’}}
However according to gpt-4o vision documentation: " Images can be passed in the user , system and assistant messages" https://platform.openai.com/docs/guides/vision
Moving back to gpt-4-turbo eliminates these errors.
How can I pass in image_urls generated by the assistant in message history?