GPT-3.5 API is 30x slower than ChatGPT equivalent prompt

I have reported this post to OpenAI help, but I’ve got only a standard answer. (as expected)

Several of my accounts are slow as hell. Only one of my spare accounts has normal speed, so I’m using the good one as a last resort.

I could only guess that some of accounts are being assigned to crowded nodes. (maybe deliberately?)

Slow ones generate at a human-readable speed. If you’re streaming, it’ll be at least bearable for users. But if you’re not streaming, your services should be as good as dead. I mean, who’s going to wait 30~50 seconds with no output? Users’ll cancel and go away, but you are still billed for the tokens.

2 Likes