Throttling on 3.5-Turbo API

Are other people experiencing the massive throttling we’ve had lately? Our application does realtime interactions and we are constantly having calls get hung for 30 seconds or more. Problem started about 3 weeks ago. I thought it might be the outage they had a couple weekends back but it persists. Less but definitely still there.

Haven’t really experienced “throttling” (like payment tier-1 users who get slower token production).

If you use streaming, you can use a watchdog and close() and restart if you aren’t getting anything from the API after 5 seconds or a bit more for GPT-4.

It is likely that all output is going through a content_filter AI, so there is added delay that sometimes may just not work.

Thanks _j. I’ll try and implement.