GPT-4 API slow response over 60sec

API
1

Hi,
I have problem with response time with GPT-4, I set timeout=60 for API, but it’s need even more, anyone had similar problem or solution ?
Is any option on API that you can monitor what %% of work is done ?

2 Likes
2

Hi,
To the best of my knowledge, there isn’t currently a way to track the progress of a request in real-time using the OpenAI API. I understand that it would be helpful to have such a feature !
Sorry for not being able to help you more…

3

Thank you for your response. Yesterday morning, it was working fine, just a bit slower. However, after 4 PM, it stopped working entirely, and it’s still not working today. The response is so slow that it’s essentially nonfunctional. Has anyone else encountered this issue?

4

Same here. GPT-4 API-Request above ~1000 tokens don’t finish at all (even after 15mins). Playground (with GPT4) and ChatGPT (with GPT4) works fine.

5

I had a similar problem. Here is what I performed to solve this issue.

  1. I used a tool called Postman to test the response time of the OpenAI server.
    (This is to compare the speed of my back-end, and my local).
    *I made sure to use the exact same prompt that my slow AI uses because shorter completion will produce faster response times, which will lead to different testing results.

  2. However, the response time was still slow in my case. Therefore, I turned on the “stream” option and set it up in the back-end. With this option, at least I could check the progress of the outputs, similar to how ChatGPT works. So, I am okaysh now. :smile:

6

Thanks. Streaming really helps. There must be some HTTP timeout in the library I was using, which is isn’t surfaced despite my settings.