Extremely disappointed with how the current API is functioning and the help centre response. Over the last 3 days, the API has been acting weirdly. Occasionally the API functions normally, and occasionally the API starts generating requests using 9 to 14 million tokens (this is despite the fact that I have set up a max count of 500 tokens in the API). I am attaching the screenshot of the requests below for reference. Is any one else facing similar issues? I have sent like 7-10 mails to help centre but they have been non responsive.
You can see how the normal API requests from mine work and how the other abnormalities are happening.
To continue, in less than 15 mins, 75 million tokens were utilised. Which is extremely impossible given the prompt limit and the token limit set.
Yeah, that doesn’t look normal.
Did you accidentally leak your key, maybe? Any way to change your code to use a new key?
Sending 7+ emails might make the problem was if it’s not in a thread and they get 10+ messages from you.
Wish I could hope more.
Hi,
The key is being sent from google firebase server, so I don’t think that leak could be an incident. Also because I change the key on a daily basis. I have been trying to contact the team to help solve this issue, as it has happened a couple of times now. And I have ended up spending over $1500 already.
In the short term, to eliminate this, they updated the response to include the tokens used. You can record this for each response and integrate it over time. If the number of tokens exceeds X per hour/minute/etc, you should throttle that user or the entire system.
3 Likes
I have the same problem at the moment.
Every 5 minutes the tokens are/were increasing. Now my usage limit is reached.
Any way to solvce this ?
