My APIs are failing after working for few hours, likely hitting rate limit other than per limit. I see the mention of daily and monthly limit but can’t find what are my limit. I am on tier 2 paid version with auto credit enabled and well within monthly budget limit (56 %).
In tier 2, the only daily-limited model should be the GPT-4 vision AI:
Model | RPM | RPD | TPM |
---|---|---|---|
gpt-4-vision-preview |
100 | 1,000 | 20,000 |
You can see what is set on all models of your account here
If you click “show all models”, that is how you will see the vision model and RPD for the model.
If you are using any other model, it is one of the per-minute limits that may be causing a hold-back.
You can also record the headers of your request, where there are runtime limits for tokens and requests per minute given along with the base.
Then finally, Assistants itself has silly limits if using that endpoint, that basically prevent you from using it normally, like polling once per second for a single job in progress is about the limit.
About a month ago they removed the RPD and TPD limits for most models.
Also noticed a disparity from a week ago: they are putting the token squeeze on -instruct, across the board:
gpt-3.5-turbo-instruct-0914
90,000 TPM
3,500 RPM
It is no longer in “default for all other models”
"rpm": [3, 3000, 3000, 3000, 3000, 3000],
"rpd": [200, 1e9, 1e9, 1e9, 1e9, 1e9],
"tpm": [150000, 250000, 250000, 250000, 250000, 250000]
Yeah, I suspect they are spinning down 3.5-instruct
nodes as it’s probably a very niche model.
Which does sadden me as it further diminishes my hopes of 3.5-instruct
fine-tuning or the creation of a 4-instruct
model.
Honestly, I think most of my niche needs could be met with a fine-tuned 4-instruct
.
Ah well…
I guess I’ll just need to wait for general availability of gpt-4
fine-tuning and for another big price cut.