Concatenated streaming packages in API response (recent change & simple fix) [svelte/javascript]

API
, , , ,
1

Hi all,

Disclaimer: I’m not the best Javascript dev, so feel free to contribute any improvements for anyone having the same issue

Recently a Svelte app I use started having some issues parsing out the JSON packages received from the API. After some light debugging I noticed that some of the received streaming messages were getting concatenated, which prevented them from being correctly parsed by JSON.parse().

image
image1528×634 69.6 KB

If you are running into a similar issue, I hope this saves you some time. A fix that worked for me was just making sure to separate any concatenated stream messages using regular expressions, and then processing as you regularly would. If this doesn’t quite fix it for you, at least now you know what to ask chatgpt about, or a possible thing to look out for.

Code before

let source = new SSE("https://api.openai.com/v1/chat/completions", {
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${DEFAULT_API_KEY}`,
    },
    method: "POST",
    payload: JSON.stringify({
      model: $gptModel.code,
      messages: msg,
      stream: true,
    }),
  });
  
source.addEventListener("message", (e) => {
	if (e.data != "[DONE]") {
	  let payload = JSON.parse(e.data);
	  let typing = false;
	  let text = payload.choices[0].delta.content;
	  // whatever processing you use for generated text
	}

Code after

let source = new SSE("https://api.openai.com/v1/chat/completions", {
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${DEFAULT_API_KEY}`,
    },
    method: "POST",
    payload: JSON.stringify({
      model: $gptModel.code,
      messages: msg,
      stream: true,
    }),
  });
  
source.addEventListener("message", (e) => {
	if (e.data != "[DONE]") {
		// console.log(e.data);
		// Split the data using a regex that detects boundaries between JSON objects
		let splitData = e.data.split(/(?<=})(?={)/);
		splitData.forEach(data => {
			try {
				let payload = JSON.parse(data);
				let typing = false;
				let text = payload.choices[0].delta.content;
				if (text == undefined) {
					typing = !typing;
				}
				if (text != undefined) {
					// whatever processing you use for generated text
				}
			} catch (error) {
				console.error("Error streaming response. Error:", error);
			}
		});
	}