O1 - Rate Limit and function calling

Hi,

excited to try out o1 via API with function calls I found that there’s a big snafu. You are limited to one request per minute on the function calls, even on the chat completion endpoint.

Now here’s why this is a problem: if the first response is a tool call, then how are you supposed to submit it?

If I submit it right away, attached to a new message and a new completion request, I immediately get the rate limit message:

Rate limit reached for o1 in organization org-drLTAb15KoZkvvfFXyYupIfA on requests per min (RPM): Limit 1, Used 1, Requested 1. Please try again in 1m0s. Visit https://platform.openai.com/account/rate-limits to learn more.

So how is that supposed to work? Shouldn’t there be at least an exception for submitting tool call results?

Is there some information in the headers maybe how soon I could submit the result?

4 Likes

One simple solution is to combine o1 with another model like gpt-4. Think about where you really need o1’s capabilities and where GPT-4 can handle things. For example, you can use GPT-4 for the initial step of receiving input an generating the function call, run the function, and then pass the results to o1 to process and analyze. This way, you use each model where it works best and avoid O1’s rate limits.

1 Like

I found that there is rate limit information in the header. So I am waiting until a new request is available, which is up to a minute to send my function call results.

Yeah, this is a totally unhelpful rate limit. 1 RPM has just been enough to show me that I really could benefit from a reasoning model, without actually giving me useful access to one. Surely I will now have to explore other providers.