I can confirm, this probably started within the last 24h, all of our previously working code (working for months) has started having a problem with the first couple of chunks.
We are talking about streaming mode here, so:
const chatResponse = await openai.createChatCompletion({
...options,
messages,
stream: true
}, {
responseType: 'stream'
})
and then handling the stream (as per lots of example code - this has been working for a long time):
chatStream.on('data', (chunk: Buffer) => {
const messages = chunk
.toString('utf8')
.split('\n')
.filter(m => m.length > 0)
for (const message of messages) {
... // skipped code
const data = message.slice(dataMarker.length)
... // skipped code
const completion: CreateChatCompletionStreamResponse = JSON.parse(data)
... // etc.
suddenly, the first chunk has a newline in it and results in two “messages”:
data: {"id":"chatcmpl-8J04wP33MFVY0pQoljcFiBsiBW3uC","object":"chat.completion.chunk","created":1699539226,"model":"gpt-4-0613","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}
data: {"id":"chatcmpl-8J04wP33MFVY0pQoljcFiBsiBW3uC","object":"chat.completion.chunk","created":1699539226,"model":"gpt-4-0613"
the first parses to “” the second fails to parse as it is not valid JSON (it’s only half the object)
then the next chunk continues:
,"choices":[{"index":0,"delta":{"content":"My"},"finish_reason":null}]}
which does not start with the data marker 'data: ’ so is ignored by code (like How to use stream: true? · Issue #18 · openai/openai-node · GitHub - which is pretty much the code we have been running for months)
the next and subsequent chunks come in as usual eg:
data: {"id":"chatcmpl-8J04wP33MFVY0pQoljcFiBsiBW3uC","object":"chat.completion.chunk","created":1699539226,"model":"gpt-4-0613","choices":[{"index":0,"delta":{"content":" journey"},"finish_reason":null}]}
It’s easy enough to mitigate probably, but why has this happened? Will it be reverted? One other issue opened about it: createChatCompletion Regression - payload is split across 2 chunks
