OpenAI API Vision model response time unstable and sometimes slow

API
,
1

I use the OpenAI API with the following request types:

  • "type": "text"
  • "type": "image_url"

However, I notice that the response time varies significantly. For example:

  • Duration till response: 5833 ms | Request size: 224 KB
  • Duration till response: 2727 ms | Request size: 475 KB

Sometimes, it even takes up to 8 seconds to get a response.

Why is the duration so unstable? What factors affect the response time, and is there anything I can do to optimize it?