Batch Processing with Rowy / GCP / Firebase

API
, ,
1

I am at a loss so wanted to start here. I have used a template by Rowy (Parallel GPT) that basically lets me build a spreadsheet with JavaScript in the columns to have Open AI API integrated.

My issue, by nature of all this, may extend into GCP / Firebase. But I am not sure if it is something with Open AI or if it even could be?

Each row of my sheet has 4 columns that use Open AI. Each of the 4 uses a different API and each only renders once the prior one finishes (to help with limits). Despite that, even in a small batch of 100 rows - The scripts only finished about 10% and then left the rest blank.

I am able to rerun the formula manually and got to about 30 today, but I just for the life of me can’t figure out what limit I am reaching based on all the information I have read on the limits page.

I am using gpt-4-vision-preview specifically for 1 of those 4, which SEEMS to be the main barrier. I know it is very limited now, however today - as an example - I have only 42 / 100 requests according to the Usage page. I have tested the token usage of each row and it is around 1,500/request.

I know I can only do 10K RPM, but even with that in mind - I have tried doing nothing for 1-2 minutes, then re-running just ONE cell and it still won’t run.

Does anyone have any insights into this?

P.S - Am I correct in assuming if all 4 cells are using 4 different API’s, that they are each then only subject to the API limits of that API? I did that since Vision was only needed for 1 of the 4 and has the harshest limits.

UPDATE - I have added better error logging and while the errors are not consistent, the one that stands out is the API reporting back “Error: 429 Rate limit reached for gpt-4-vision-preview in organization org-79etNiGwugRMHREd7xFDb3qA on requests per day (RPD): Limit 100, Used 100, Requested 1. Please try again in 14m24s. Visit https://platform.openai.com/account/rate-limits to learn more.”

I have checked my usage page again for the API being called (gpt-4-vision-preview) and even if I total up the Requests over the last TWO days (understanding the rolling window basis), it is still under 100?

2

Hi and welcome to the Developer Forum!

It is more likely you are hitting one of the other Rate limits as new accounts are on a low Tier until credit has been added and a number of days passed as can be seen here

image
image698×397 17.2 KB

https://platform.openai.com/docs/guides/rate-limits/usage-tiers

It may be beneficial to contact the original creator of the script/plugin to discus this issue with them as this is not an application developer by OpenAI themselves.

Happy to help you debug your issues but it may require some technical knowledge on your part to make use of.

3

Thanks for the input! I have verified I am on Tier 1 and that Vision limits really for ALL Tiers is 100 RPD.

I have had my account funded for over 48 hours.

My current course of action is to wait a full 48 hours to let all requests made clear out of the limits and my account to be that much older and try running my scripts again.

But it it still odd that it is claiming I reached 100 RPD on Vision when the Usage page shows me under 100 for the last 2 days (it currently shows 80 requests made yesterday and the day before).

4

I do want to update that I’ve waited 48 hours, done a solo test to verify how many tokens my request is, and then was successful in running 59 requests by throttling to just 9 requests per minute.

I will test further but I think all is working and I just had to add a Firestore document to externally have a counter variable that Rowy could read and let me set a throttle mechanic.