createChatCompletion Regression - payload is split across 2 chunks

API Bugs
, ,
1

Ever since a few hours ago we’re seeing a different behavior with createChatCompletionRequest streaming api which was not present before.

The 2nd payload is always split into 2 chunks:

first chunk :

payloads [
  'data: {"id":"chatcmpl-8IaMjS68fE6eEtFvAuzQloHnx1yAd","object":"chat.completion.chunk","created":1699440385,"model":"gpt-4-1106-preview","system_fingerprint":"fp_a24b4d720c","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}',
  'data: {"id":"chatcmpl-8IaMjS68fE6eEtFvAuzQloHnx1yAd","object":"ch'
]

2nd chunk holds a continuation of the 2:

payloads [
‘at.completion.chunk",“created”:1699440385,“model”:“gpt-4-1106-preview”,“system_fingerprint”:“fp_a24b4d720c”,“choices”:[{“index”:0,“delta”:{“content”:“```”},“finish_reason”:null}]}’,
‘data: {“id”:“chatcmpl-8IaMjS68fE6eEtFvAuzQloHnx1yAd”,“object”:“chat.completion.chunk”,“created”:1699440385,“model”:“gpt-4-1106-preview”,“system_fingerprint”:“fp_a24b4d720c”,“choices”:[{“index”:0,“delta”:{“content”:“json”},“finish_reason”:null}]}’,
‘data: {“id”:“chatcmpl-8IaMjS68fE6eEtFvAuzQloHnx1yAd”,“object”:“chat.completion.chunk”,“created”:1699440385,“model”:“gpt-4-1106-preview”,“system_fingerprint”:“fp_a24b4d720c”,“choices”:[{“index”:0,“delta”:{“content”:“\n”},“finish_reason”:null}]}’,
‘’
]

I didn’t see any reference this behavior, is this expected? the weird thing is that it’s only the 2nd payload.

1 Like
2

I don’t have access to chat Completion
I’ve tried to upgrade OpenAI in my terminal but still its version doesn’t change its still 1.1.1

3

Can confirm. This started for us about 1 day ago, in all streaming code that has been deployed just fine since probably January.

See Malformed streaming answers from GPT-4 completions API lately

The first chunk is unusual in that it has two 'data: ’ payloads separated by a newline (which is fine, though I am not sure we ever saw that in practice), but the second payload is only half the completion object. The second chunk is the rest of the object, but is invalid because it does not start with 'data: '. Then the rest of the stream is fine: a single 'data: ’ payload in each chunk

1 Like
4

as a workaround: I ended up upgrading our open ai npm package which overcomes this issue.

5

I’m getting this same issue right now, tried upgrading npm package but this didn’t help. Did anybody find another workaround?