Malformed streaming answers from GPT-4 completions API lately

API
4

I can confirm, this probably started within the last 24h, all of our previously working code (working for months) has started having a problem with the first couple of chunks.

We are talking about streaming mode here, so:

	const chatResponse = await openai.createChatCompletion({
		...options,
		messages,
		stream: true
	}, {
		responseType: 'stream'
	})

and then handling the stream (as per lots of example code - this has been working for a long time):

			chatStream.on('data', (chunk: Buffer) => {
				const messages = chunk
					.toString('utf8')
					.split('\n')
					.filter(m => m.length > 0)
				for (const message of messages) {
					... // skipped code
					const data = message.slice(dataMarker.length)
					... // skipped code
					const completion: CreateChatCompletionStreamResponse = JSON.parse(data)
					... // etc.

suddenly, the first chunk has a newline in it and results in two “messages”:

data: {"id":"chatcmpl-8J04wP33MFVY0pQoljcFiBsiBW3uC","object":"chat.completion.chunk","created":1699539226,"model":"gpt-4-0613","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-8J04wP33MFVY0pQoljcFiBsiBW3uC","object":"chat.completion.chunk","created":1699539226,"model":"gpt-4-0613"

the first parses to “” the second fails to parse as it is not valid JSON (it’s only half the object)

then the next chunk continues:

,"choices":[{"index":0,"delta":{"content":"My"},"finish_reason":null}]}

which does not start with the data marker 'data: ’ so is ignored by code (like How to use stream: true? · Issue #18 · openai/openai-node · GitHub - which is pretty much the code we have been running for months)

the next and subsequent chunks come in as usual eg:

data: {"id":"chatcmpl-8J04wP33MFVY0pQoljcFiBsiBW3uC","object":"chat.completion.chunk","created":1699539226,"model":"gpt-4-0613","choices":[{"index":0,"delta":{"content":" journey"},"finish_reason":null}]}

It’s easy enough to mitigate probably, but why has this happened? Will it be reverted? One other issue opened about it: createChatCompletion Regression - payload is split across 2 chunks