How to chat with an AI-Generated image via responses api?

API
1

I want to let me users generate an image and then further communicate about it.

However, when I try, the api informs me that ‘output_text’ and ‘refusal’ are allowed. If I change the role to “user”, it works, so this seems to be blocked for the assistant role.

Is this a known limitation? I could not find anything in the docs. If yes, can I work around it?

Full Response:

{
    "error": {
        "message": "Invalid value: 'input_image'. Supported values are: 'output_text' and 'refusal'.",
        "type": "invalid_request_error",
        "param": "input[1].content[0]",
        "code": "invalid_value"
    }
}

Full Request:

{
    "model": "gpt-5",
    "stream": false,
    "tools": [
        {
            "type": "image_generation"
        }
    ],
    "input": [
        {
            "role": "user",
            "content": [
                {
                    "type": "input_text",
                    "text": "Generate an image of a nice suburb house. Do not ask for specifications, just create one."
                }
            ]
        },
        {
            "role": "assistant",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "data:image/jpeg;base64,....b64img here}"

                }
            ]
        },
        {
            "role": "user",
            "content": [
                {
                    "type": "input_text",
                    "text": "Describe this images style"
                }
            ]
        }
    ]
}
2

This is a known limitation.

The AI assistant you are chatting with is not the one that generated image data.

Simulating the AI model “producing” a vision component could give it wrong ideas about what it can generate as further output.

On the Responses API, you can return an image as a tool return as a result of the output calling a tool, with a matching pair of tool IDs.

3

Thanks, that was helpful!