The gpt-4 model streaming fails, but other models work

API Bugs
1

Hello,
Starting this morning we experienced errors while streaming.
Also the example code from the OpenAI Streaming page works with model gpt-4-1106-preview but it fails for model gpt-4
While debugging, we received this error " The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error.'"
Is anybody else experiencing this issue?
Thank you!

5 Likes
2

Also experiencing this since yesterday, identical issue.

Using the Python SDK, with steaming on we get “An error occurred during streaming”
Using the playground, I get the same error message as the OP.

3

Experiencing the same issue, but only for one of my client’s API keys. I have other clients whose API keys are still working fine for gpt-4 streaming.

For the one that’s broken I see the same pattern as everyone else. Non-streaming chat endpoint works for gpt-4 and gpt-4-0613, but streaming fails. Streaming and non-streaming is fine for gpt-4-1106-preview.

# broken for gpt-4 streaming

curl https://api.openai.com/v1/chat/completions   -H "Content-Type: application/json"   -H "Authorization: Bearer sk-XXXXXXXXXXX"   -d '{
  "model": "gpt-4",
  "messages": [
      {
        "role": "system",
        "content": "just say hi"
      }
    ],
    "stream": true
  }'


# broken for gpt-4-0613 streaming

curl https://api.openai.com/v1/chat/completions   -H "Content-Type: application/json"   -H "Authorization: Bearer sk-XXXXXXXXXXX"   -d '{
  "model": "gpt-4-0613",
  "messages": [
      {
        "role": "system",
        "content": "just say hi"
      }
    ],
    "stream": true
  }'



# fine for gpt-4 without streaming

curl https://api.openai.com/v1/chat/completions   -H "Content-Type: application/json"   -H "Authorization: Bearer sk-XXXXXXXXXXX"   -d '{
  "model": "gpt-4",
  "messages": [
      {
        "role": "system",
        "content": "just say hi"
      }
    ]
  }'

# fine for gpt-4-0613 without streaming

curl https://api.openai.com/v1/chat/completions   -H "Content-Type: application/json"   -H "Authorization: Bearer sk-XXXXXXXXXXX"   -d '{
  "model": "gpt-4-0613",
  "messages": [
      {
        "role": "system",
        "content": "just say hi"
      }
    ]
  }'


# fine for gpt-4-1106-preview streaming

curl https://api.openai.com/v1/chat/completions   -H "Content-Type: application/json"   -H "Authorization: Bearer sk-XXXXXXXXXXX"   -d '{
  "model": "gpt-4-1106-preview",
  "messages": [
      {
        "role": "system",
        "content": "just say hi"
      }
    ],
    "stream": true
  }'
1 Like
4

… which are the same model.

Ran it with logprobs, timers, etc, no failures streaming with this API key.

Certain accounts being affected is certainly an oddity. We might compare tiers of those who fail - are they all in tier 1 for increased-latency-as-a-feature, for example.

Replete with splendor, she loves to…
…ᴅᴀɴᴄᴇ ᴜɴᴅᴇʀ ᴛʜᴇ sᴛᴀʀʟɪᴛ sᴋʏ, ʜᴇʀ ʟᴀᴜɢʜᴛᴇʀ ᴇᴄʜᴏɪɴɢ ᴛʜʀᴏᴜɢʜ ᴛʜᴇ ɴɪɢʜᴛ. Hᴇʀ ᴇʏᴇs sᴘᴀʀᴋʟᴇ ᴡɪᴛʜ ᴊᴏʏ ᴀs sʜᴇ ᴛᴡɪʀʟs, ʜᴇʀ ᴅʀᴇss sᴡɪʀʟɪɴɢ ᴀʀᴏᴜɴᴅ ʜᴇʀ. Eᴀᴄʜ ᴍᴏᴠᴇᴍᴇɴᴛ ɪs ᴀ ᴛᴇsᴛᴀᴍᴇɴᴛ ᴛᴏ ʜᴇʀ ʟᴏᴠᴇ ғᴏʀ ʟɪғᴇ, ᴀ ᴄᴇʟᴇʙʀᴀᴛɪᴏɴ ᴏғ ᴛʜᴇ ʙᴇᴀᴜᴛʏ ᴛʜᴀᴛ sᴜʀʀᴏᴜɴᴅs ʜᴇʀ. Sʜᴇ ɪs ᴀ ᴄʀᴇᴀᴛᴜʀᴇ ᴏғ ᴛʜᴇ ɴɪɢʜᴛ, ᴀ ʀᴀᴅɪᴀɴᴛ ʙᴇᴀᴄᴏɴ ɪɴ ᴛʜᴇ ᴅᴀʀᴋɴᴇss, ʜᴇʀ sᴘɪʀɪᴛ ᴀs ʙᴏᴜɴᴅʟᴇss ᴀs ᴛʜᴇ ᴜɴɪᴠᴇʀsᴇ ɪᴛsᴇʟғ.

5

Experiencing the same issue. gpt-4 and gpt-4-0613 streaming fails. Only some API keys are affected - some organizations to be precise - another org works fine. Tried to create a new API key in problematic org - did not help.
Currently on Usage tier 4 in the org that has problem.

6

Hey folks, thank you for flagging this! We are looking into it now. If anyone has request ID’s from failed requests that would be useful.

1 Like
7

We just rolled out a potential fix, can folks give this a try and let me know if they are still seeing this error?

1 Like
8

Okay this should now be resolved, thank you all for flagging an being patient while we worked to address this!

4 Likes
9

Perfect, thanks Logan & team. Working for me now

10

It’s working now without issues. Thank you!