Hi OpenAI team,
My title basically summarizes the feedback in a nutshell: currently, there is no way to track the number of tokens you are using in the Assistants API like you can in the completions API. For many developers, this is probably the most important thing in terms of keeping costs in check and with an added level of ambiguity with retrieval, this is crucial.
I know we could probably implement a Tiktoken solution, but this is an added level of complexity and may not be 100% accurate with Tiktoken, especially if OpenAI has background system prompts that we are billed for.
Absolutely phenomenal work at Dev day and I am looking forward to the improvements in the coming weeks.
- Landon
@logankilpatrick — I hate tagging you directly but this is a fairly pressing issue for most teams right now. Appreciate the help!