Hello, I’m trying to use the omni moderation model on a fresh account (OpenAI wrongly banned me years ago after I complied with all their requests and they didn’t respond to my emails 3 times so I had to switch accounts). However, I appear to be running into a specific rate limit for 10,000 requests per day which is only about 6.94 requests per minute. I need about 10x that amount (give or take).
I would like to pay some money to increase these rate limits, but the moderation endpoint is free, so I’d just be putting money into the account I don’t need to use (there is no model on OpenAI that’s price-performance to use right now, except the free moderation model). I am willing to do this and put $50 to get to “Tier 2” after 7 days. However, there is no transparency into whether this will raise the rate limit of omni-moderation. Documentation only lists spend limits and vaguely says “most models” will increase rate limits by tier but not by how much.
I didn’t want to wait for an answer so I just went ahead and paid $40 to make the total $50 (I had paid $10 earlier). Fortunately, I can see now that I’m Tier 2, and the omni-moderation endpoint daily limit is completely removed, leaving only the 500-per-minute rate limit which is more than enough for me.
UPDATE: The dashboard now claims I’m Tier 2 with the daily limit completely removed, but I’m STILL seeing the same frequency of errors with “Too many requests”!!
What could be the reason now??
UPDATE: Unfortunately, the moderation endpoint has a tokens per minute limit of 20,000 at Tier 2. Even at higher tiers, this is not enough for people with high rates of requests. omni-moderation Model | OpenAI API
It seems like they need to make the moderation endpoint PAID instead of FREE, and charge by the input tokens similar to LLM pricing; that way people can actually use this endpoint in a reasonable manner instead of trying to pay $1,000 immediately to reach Tier 5 only to still hit limits during peak hours because even at Tier 5 the limits are only 500,000 tokens per minute, whereas in an app you might have multiple users issuing requests where each one is 32k tokens.
Dont worry - Pay and you will go to higher tiers. It is done instantly and automatically.
We just added it in our documentation just now. We had customers paying $20 to try it out. But that keeps them on tier 1 - they run out of tokens, and open a support ticket on us.
So in our docs we put a disclaimer "fund your openai account with $100 or $200” and only then talk to us if you have problems. Like that we get them on tier 3/4 immediately.
The limiting factor is “tokens per minute”, not requests per minute
As you can see even on highest tier, it only allows 500,000 tokens per minute. Compare that to the regular LLM models which allow far higher rate limits. There must be at least some Tier 5 users/developers complaining about this, no? Not to mention, it is not reasonable to expect someone to pay $1,000 up front just to be able to use a moderation API with a decent amount of traffic.
P.S. if you look at that documentation, text-moderation-latest and text-moderation-stable have much more reasonable/generous rate limits than omni-moderation. However in all their wisdom OpenAI appears to have sunset these models without warning anybody, because when trying to request them, it says the model does not exist.