Just to confirm, what happens if you make a call to the OpenAI API using one of the official SDKs, or even a simple curl request?
I have not worked with the AI SDK yet. Is there a community where similar issues are being reported?
Regardless of the outcome, I will monitor the community to see if additional reports come in.
We have been facing something very similar when calling via OpenAI SDK itself.
response = client.responses.create(**request_params)
But the response is never received although HTTP 200 OK logs comes for the same.
I’m raising this with the team so they can take a closer look. To help us escalate this properly, could you please confirm that you haven’t set unusually low timeout values on your side, and whether this is a new issue for you?
Thanks for your understanding. This information will make it easier to get the right attention on the problem, as these are among the first questions that will need to be answered anyway.
Timeouts are set to default, and yes it’s a relatively new issue that became increasingly frequent over the past ~week. I should also note that I haven’t seen the error in the past 24 hours. This was an abrupt drop-off in errors, so something may have been fixed on OpenAI’s side?
we’re also experiencing this issue — it’s been happening for a few weeks now, but was especially bad 2 days ago. In our traces the call to the openai sdk gets stuck indefinitely until it times out with the same error as OP.
The team is actively investigating this. I cannot provide a timeline at this point, but if you have any additional information to share, please do so here or via a support ticket and reference this community topic.
Hey everyone, Apologies for the inconvenience. Our team is taking a look into this issue. We will provide an update as soon as we identify the issue. Thank you!
Steve here from the OAI eng team. My understanding of BodyTimeoutError is that it is the result of a timeout waiting for data on the wire between events. Depending on the nature of your integration, it may be many seconds in between events. Would folks be willing to bump up bodyTimeout to 1 minute and see if this resolves the issue?
In the meantime, we are shipping a new feature which will push keepalive events into the stream at a regular interval to make sure this “just works” for most clients
Thanks for following up Steve! Our timeout is set to 5 minutes and we still often see this issue — is this expected? I don’t know if increasing our timeout value will resolve this.
heartbeat: int - seconds - sets an interval where whitespace is sent before a json output begins being produced in a non-stream setting, verifying ongoing “thinking” and token production.
Or better, chunks of spaces transmitted for every token, for those who really want to evaluate the quality of unseen generation service.
You can test such an idea with a local proxy to the API and a short timeout on a client. Simulate a cloud worker platform that terminates inactive connections after 60s. See Chat Completions not drop despite the long reasoning. See whos client breaks by optional parameter and where in the stack you’d need to integrate this.
Responses has the facility for an ignorable event when no summary is requested.
As additional context: we observe this issue much more frequently when we are streaming API responses. This also occurs with non-streamed responses, but the occurrence rate is significantly lower.