Thread token usage endpoint

[yga]

Any way for me to check out how many tokens a given thread is using
(in the context of the Assistants API)

Or is it all abstracted by the assistant - model interaction and there is no way around it?

[jorgeintegrait]

This is a common question that the community still hasn’t found an answer to. At the moment, it is abstracted and you can only estimate it, if you’re interested on that I have a couple recommendations here:

Impact of Instruction Size and Thread Length on Token Usage in OpenAI Assistant

There isn’t clear information on assistant token usage yet. Not on the documentation and not on this forum I’m afraid. Still, it would be natural that the pre-set instructions will consume tokens per message, and that the more tools you use (dalle, retrieval, code interpreter) the longer these instructions are. You can try starting a new assistant with gpt-3.5 with all the tools and no additional instructions and ask it to give you the instructions is has verbatim. This can give you an estimate…

[yga]

I see, the tough part is when some functions are called and the assistant uses only a few parts of the information for the completion. It’s becoming a guessing game

[yga]

Small update
I’ve been adding messages and runing fnuction calls, and for functions with larger results, there seems to be a limit of 10000 tokens added to the context per function call, or close enough

I’m using tiktoken btw