Hi, I am calling GPT4o and Vision through Zapier for a variety of functions, both image and non-image related. I have calculated that each run uses around 20k tokens based on what Zapier tells me. I have calculated this based on by adding up what the API calls come back with. When I look at the usage from a single run in the Assistant Dashboard I am charged for over 800k, sometimes over 1 million tokens per run. Why might this be? Are there additional tokens being used apart from the input and output tokens? Thank you!
1 Like
Assistants API performs internal functions and iterations over which you don’t have direct control. These consume tokens too.
If you are concerned about keeping close control over token budgets I would recommend shifting to the Completions API and implementing your own code to manage the “conversation”
There’s been quite a lot of discussion on this already:
https://community.openai.com/search?q=assistants%20api%20token%20usage
2 Likes