Hey wondering if anyone has encountered this.
I’m streaming from GPT3 via PHP (so it loads the content token by token, which I can then stream to the browser)
Works great, but I notice there are often missing words, which I assume are missing tokens.
Does anyone know what might be causing this? Has anyone else experienced this?
Very late response - but I’m also encountering this on 3.5 chatCompletions streaming (via node/js)
If anyone has a fix - I’d love to hear it!
haven’t noticed this but i’m wondering what SDK you’re using for streaming? I’m trying to get this working also for PHP…
The same thing happens with C#. I coded around it.
I thought nothing was coming the first time I saw it and aborted the rest of the connection.
@raymonddavey Could you explain what you mean when you say you coded around it? Are the missing tokens present but being parsed out?
Nothing to clever in the end. It was just a bit confusing the first time I got empty records at the start
Once I realized what was happening I ignored the empty record and continued until I got [DONE]
Before that I was stopping when I got a blank and thought it wasn’t working
I have the same problem when using stream on Dart/Flutter, here are some of my encounters:
System Prompt: You are a helpful assistant
Input: What is the largest mammal in the world
Response: The mammal the the whale It of to98 feet (30) and over 200 tons
Input: What is the tallest building in the world
Response: the, but foundation for only can build your tall as Bur Khalifa
The response has missing/skipping words.