Hi, I just started seeing 7-8 seconds latency in retrieving run info on v1/threads/{thread_id}/runs API.
Waiting for the response by GPT to be generated is not a big deal as it is expected but I was depending on calling this API polling style(once per second) to check if the output is ready and now I see that this is not workable.
Are there any performance issues or rate limits for checking the runs API? Any ideas on how to fix this?
Performance of the Assistant API has been absolutely awful in the last 2 days. You sometimes have to wait 2 to 3 minutes for a run (Code Interpreter) to complete. Same thing was taking maybe 30 seconds the week before.
It’s not documented but I think they are throttling API users if you haven’t paid some money in.
I noticed that in the Completion API headers it mentioned rate limit reset 8s and my requests were taking about 8 seconds no mater the request. That particular header wasn’t documented so I suspected that I was getting throttled. The funny thing though, the completion API wasn’t slow, only the Assistant API.
So your credit doesn’t expire but you get VERY SLOW responses after some time. I paid in some money and my API response speeds are restored.
I haven’t released my app yet, so the initial 5$ were enough for testing for more than a month or two, so I guess I hit the throttlegate.
I am in Tier3 and have already paid several hundred dollars in the last 2 months…
If OpenAI has such policy then they should be transparent about it. I am not saying that they do but such a disparity in performance over time is concerning.
Check this unofficial API status page if you are seeing performance issue if it coincides to the graphs
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.