As usage grows, API costs can add up quickly. I’m curious how other developers are tracking, limiting, or optimizing token usage in production.
Are there any proven strategies or tools you recommend to keep costs predictable while maintaining good output quality?
2 Likes
- Dashboard
- Unique key for specific site or site function to make costs/trade-offs more transparent and obvious
- Use of Completions end point to make sure you are the one deciding on the number of iterations and tokens being consumed, not handing that discretion over to the company making the money from the number of tokens and iterations.
- Batches & lower service quality calls
- Automation (batch processing like sidekiq) to auto retry failed calls to optimise workflow and prevent you starting from scratch
- Don’t use reasoning unnecessarily
- Pick a model that is good enough, not necessarily the latest and greatest (and most expensive!)
- Look at the population of what you are processing and drop entities that are of low quality/don’t add significant value
4 Likes