How to cap cost to prevent API cost from blowing up?

API
1

There is a basic concern I have that some eventual bug/poor coding may lead to an infinite loop of GPT API calls that will blow a hole in my finance and I may not realize it until it is too late. So I was curious how people prevent this scenario.

I know in AWS you can set up a monthly cost/budget, where it will alert you if you hit [x%] of budget. But of course, I run the risk of missing the alert or it could be too late by the time I see the alert. So I was wondering how people deal with this potential scenario?

(to be clear - I am not talking about cost optimization. I just want some way to prevent cost from ‘blowing up’ due to an infinite loop or something of that nature)

I was wondering if there are some type of cost safeguards I could put in place.

2

This is an understandable concern. You can set both budget alerts and monthly hard limits for spend at https://platform.openai.com/settings/organization/limits . I’m not aware if you can get more granular than that though (ie, limits per project/api key), but the global limit can prevent the worse bugs.

2 Likes