GPT 4 Lightning Fast Now?!

In the last hour I noticed that gpt 4 in chatgpt is literally generating paragraphs in less then a second!?

Im not finding anything anywhere on this. Is this a new update?

Is it this fast on the apis?

Not fast on API. I get the usual spurious plodding text to -0613 and -0314, that looks like the average of 15 tokens per second. Then the second call to the current model from my performance measurement script is sitting here hanging with no tokens being produced.

Fast tokens is not a good sign. You could have been moved to a H100 machine with a testing version of the model with low concurrent users – or they could be testing further efficient model degradation to see how objectionable low computation is to users.

1 Like