Hello,
I have tried a payd billing plan for the API. But I am wondering, that the responses are so slow. I wait 10 sec for a call of the API.
I want to call the API at least 10000 times to get an LLM written korpus of sentences. But with this tempo it last one week!
Use those numbers as a guide you can add credit to your API account and wait the specified period to gain limit increases.
There does not seem to be an option for that at the moment, I imagine that will happen when more compute is available.
My RPM/TPM for gpt-4 was finally reset the same as the tier shows. They may just be running some process that is still updating accounts.
I don’t know if I would mess with the new button for monthly accounts “add credit” without knowing any side effects to the account status.
@simonchatgpt3 Have you tried gpt-3.5-turbo-instruct? It’s a completion model and you might need to modify your prompt a bit. Depends on the scenario, its TPS is at least 2x-3x higher than the 3.5 chat completion models, which means it can save you at least half of the time.
The responses being slow doesn’t preclude you from batching dozens of them at a time. You don’t need to wait for the last API call to finish in a loop, and you can write code that runs right at your rate limit, retrying if you get denied by the API for rate.
