Dropped tokens when streaming via PHP?

API
1

Hey wondering if anyone has encountered this.

I’m streaming from GPT3 via PHP (so it loads the content token by token, which I can then stream to the browser)

Works great, but I notice there are often missing words, which I assume are missing tokens.

Does anyone know what might be causing this? Has anyone else experienced this?

Thanks

2

Very late response - but I’m also encountering this on 3.5 chatCompletions streaming (via node/js)
If anyone has a fix - I’d love to hear it!

3

haven’t noticed this but i’m wondering what SDK you’re using for streaming? I’m trying to get this working also for PHP…

4

I am implementing a chat stream using the native JavaScript fetch api for the ‘https://api.openai.com/v1/chat/completions’ endpoint, and I am also seeing dropped tokens, however it is just the first token that is consistently absent.

5

The same thing happens with C#. I coded around it.

I thought nothing was coming the first time I saw it and aborted the rest of the connection.

1 Like
6

@raymonddavey Could you explain what you mean when you say you coded around it? Are the missing tokens present but being parsed out?

7

Nothing to clever in the end. It was just a bit confusing the first time I got empty records at the start

Once I realized what was happening I ignored the empty record and continued until I got [DONE]

Before that I was stopping when I got a blank and thought it wasn’t working

2 Likes
8

I have the same problem when using stream on Dart/Flutter, here are some of my encounters:

Model: gpt-3.5-turbo
System Prompt: You are a helpful assistant

Example #1
Input: What is the largest mammal in the world
Response: The mammal the the whale It of to98 feet (30) and over 200 tons

Example #2
Input: What is the tallest building in the world
Response: the, but foundation for only can build your tall as Bur Khalifa

The response has missing/skipping words.