Hi,
I don’t need help with optimising the prompt itself, but to understand what’s the most recommended way to parallelise (if this is allowed) the calls to the API so that the task completes a little faster.
Also, could anyone confirm if there is a limit to the number of concurrent calls? Some information I’ve found on the internet suggests that only up to two concurrent requests are possible.
Any other solution that could help complete the task faster with respect to making the API requests would be appreciated. Thanks!
1 Like
Hey, welcome to the forum.
There’s no way to send multiple calls at the same time, as far as I know.
There’s also rate limits in place.
https://help.openai.com/en/articles/5955598-is-api-usage-subject-to-any-rate-limits
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to handle rate limits\n",
"\n",
"When you call the OpenAI API repeatedly, you may encounter error messages that say `429: 'Too Many Requests'` or `RateLimitError`. These error messages come from exceeding the API's rate limits.\n",
"\n",
"Rate limits are a common practice for APIs, and they're put in place for a few different reasons.\n",
"\n",
"- First, they help protect against abuse or misuse of the API. For example, a malicious actor could flood the API with requests in an attempt to overload it or cause disruptions in service. By setting rate limits, OpenAI can prevent this kind of activity.\n",
"- Second, rate limits help ensure that everyone has fair access to the API. If one person or organization makes an excessive number of requests, it could bog down the API for everyone else. By throttling the number of requests that a single user can make, OpenAI ensures that everyone has an opportunity to use the API without experiencing slowdowns.\n",
"- Lastly, rate limits can help OpenAI manage the aggregate load on its infrastructure. If requests to the API increase dramatically, it could tax the servers and cause performance issues. By setting rate limits, OpenAI can help maintain a smooth and consistent experience for all users.\n",
"\n",
"Although hitting rate limits can be frustrating, rate limits exist to protect the reliable operation of the API for its users.\n",
"\n",
"In this guide, we'll share some tips for avoiding and handling rate limit errors."
]
show original
If you need to access more frequently, I would reach out to OpenAI chat support and ask…
https://help.openai.com/en/
Hope this helps!
