lachlan
February 18, 2023, 6:36am
1
Hey wondering if anyone has encountered this.
I’m streaming from GPT3 via PHP (so it loads the content token by token, which I can then stream to the browser)
Works great, but I notice there are often missing words, which I assume are missing tokens.
Does anyone know what might be causing this? Has anyone else experienced this?
Thanks
Very late response - but I’m also encountering this on 3.5 chatCompletions streaming (via node/js)
If anyone has a fix - I’d love to hear it!
haven’t noticed this but i’m wondering what SDK you’re using for streaming? I’m trying to get this working also for PHP…
I am implementing a chat stream using the native JavaScript fetch api for the ‘https://api.openai.com/v1/chat/completions ’ endpoint, and I am also seeing dropped tokens, however it is just the first token that is consistently absent.
The same thing happens with C#. I coded around it.
I thought nothing was coming the first time I saw it and aborted the rest of the connection.
1 Like
@raymonddavey Could you explain what you mean when you say you coded around it? Are the missing tokens present but being parsed out?
Nothing to clever in the end. It was just a bit confusing the first time I got empty records at the start
Once I realized what was happening I ignored the empty record and continued until I got [DONE]
Before that I was stopping when I got a blank and thought it wasn’t working
2 Likes
I have the same problem when using stream on Dart/Flutter, here are some of my encounters:
Model: gpt-3.5-turbo
System Prompt: You are a helpful assistant
Example #1
Input: What is the largest mammal in the world
Response: The mammal the the whale It of to98 feet (30) and over 200 tons
Example #2
Input: What is the tallest building in the world
Response: the, but foundation for only can build your tall as Bur Khalifa
The response has missing/skipping words.