Completions API with streaming closes connection before ending the stream

API Bugs
1

When using the completions API with streaming, I occasionally get a “Premature close error” that occurs mid-stream. It can occur at the start or at the end. It seems like the prompt and model do not affect the issue. It occurs in about 5–10% of requests. Tier-level rate limits are not reached. For some reason, the stream is closed from the server side. Example of request:

{
"endpoint": "/v1/chat/completions",
"frequency_penalty": 0,
"max_tokens": 16383,
"messages": [
{
"content": "request content",
"role": "user"
}
],
"method": "POST",
"model": "gpt-4o-mini",
"n": 1,
"presence_penalty": 0,
"stream": 1,
"temperature": 1,
"top_p": 1
}

Requests are carried through Langchain framework.

The image shows an error message related to a "Premature close" in a Node.js application using the node-fetch module, with a stack trace detailing the error's origin. (Captioned by AI)
The image shows an error message related to a "Premature close" in a Node.js application using the node-fetch module, with a stack trace detailing the error's origin. (Captioned by AI)1330×342 75.2 KB

2

try incressing the fetch timer or changing the qrequency_penalty to somewhere between -2 and 2