Hello everyone,
I would like to know how to stop streaming create chat completion. According to the documentation of OpenAI Node API library ([OpenAI Node API Library]) it is necessary to do you can break from the loop or call stream.controller.abort()
But I have the impression that this doesn’t seem to work at all. He still continues to generate.
For streaming i am using SSE
const response = await openai.chat.completions.create({
model: "gpt-3.5-turbo-16k",
messages: [
{
role: "user",
content: userMsg,
},
{
role: "system",
content: systemMsg(project)
}
],
stream: true,
temperature: 1,
max_tokens: 16185,
top_p: 1,
frequency_penalty: 0,
presence_penalty: 0,
});
io.on("connection", (socket) => {
socket.on("cancel_request", (data) => {
if (data === true) {
keepGoing = false;
response.controller.abort();
}
});
});
for await (const chunk of response) {
if (chunk === undefined) return;
res.write(`data: ${JSON.stringify(chunk.choices[0].delta.content)}\n\n`);
if(!keepGoing){
break;
}
}
From my Frontend i use Socket IO. When user click on button ‘Cancel’ i emit message and i listening on my backend.
Small clarification, my frontend is separate from the backend.
Thank you in advance for your answers
