Long response time to process completion

API
1

How long does it take for your scripts to process the API? For a simple API call, it takes at least 40 to 50 seconds. Is that normal?

I know I should cache my response for similar prompts but unfortunately in my app, there is no repeat prompt. All prompts are unique and I cant cache them. So I have to make real time live api call for each query/prompt.

Thanks in advance

1 Like
2

I have similar time values. Few weeks ago with bigger data set I could complete about 5 api requests per minute. Today for example I waited about 30 sec to run a simple test completion.