[stream, background] This response can no longer be streamed because it is more than 5 minutes old

I am streaming responses with background mode ON. Because, sometimes my connection to the OpenAI server would drop with an httpx.ReadError, httpx.ReadTimeout or httpx.RemoteProtocolError exception.

This is working but sometimes I am now getting an openai.BadRequestError with the text that is on the title.

Here’s my code:

    state = {
        "cursor": None,
        "response_id": None,
    }
    max_retries = 3

    async with RealtimeDB() as realtime_db:
        for attempt in range(max_retries):
            try:
                if state["cursor"] is None or state["response_id"] is None:
                    # First attempt, or failed before getting response_id - start fresh
                    stream = await aopenai_client.responses.create(
                        model=model,
                        input=input,
                        instructions=instructions,
                        previous_response_id=previous_response_id,
                        tools=tools,
                        reasoning=reasoning,
                        stream=True,
                        background=True,
                    )
                    async for event in stream:
                        await _handle_event(event, state, realtime_db, thought_path, realtime_path)
                else:
                    # Resume from cursor
                    async with aopenai_client.responses.stream(
                        response_id=state["response_id"],
                        starting_after=state["cursor"],
                    ) as stream:
                        async for event in stream:
                            await _handle_event(event, state, realtime_db, thought_path, realtime_path)

                return state

            except (httpx.ReadError, httpx.ReadTimeout, httpx.RemoteProtocolError):
                if attempt == max_retries - 1:
                    raise
                await asyncio.sleep(2**attempt)

Note the error happens a few times a day out of ~1000 messages a day.

OpenAI version: ‘2.15.0’

1 Like

I don’t think it’s expected that you would attempt “stream” when you put a response in “background”; it’s odd that you wouldn’t get the same immediately closed connection with object that is the deliverable in non-streaming.

The idea described behind the parameter is “I can wait to pick up the response when done; closing the connection now won’t terminate the generation”.

It would be nice to have a specific “resilient mode” that lets you stream, but the stream would not stop generation by detection of a closed connection. Then that could even work on chat completions to supply you the request ID early when streaming and when using “store”:“true” to pick up what kept running.

please read Streaming A Background Response

1 Like

Thanks for the interesting read I hadn’t encountered before.

That OpenAI wrote such an error message on the API might mean they didn’t consider delivering success and a stream that could stay open for the half-hour a model can think - that they expect you to paginate and chunk yourself with a background service not fully-implemented.

Perhaps the key understanding is: “SDK support for resuming the stream is coming soon.”

A hard pattern to patch together for having 5 minutes of streaming if there was some content produced, and then you have the client then consume the full background response.

You can also see if your symptom with normal requests is that you are encountering a hosting timeout, for example, if they cut you off at 60 seconds when there is no event transmitted by a long-reasoning model.