Instructions consume tokens, chat history consumes tokens, so does retrieval of data from your stored files, so does sending information from the code interpreter and tool calls. The GPT models are stateless, everything has to be sent to the model from scratch every time you make an API call.
Foxalabs
3
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Token use for updating instructions in an assistant | 1 | 1532 | December 16, 2023 | |
| Token Optimization for Assistants API - Excesive token count | 2 | 3172 | May 24, 2024 | |
| Impact of Instruction Size and Thread Length on Token Usage in OpenAI Assistant | 8 | 4358 | May 21, 2024 | |
| Assistant using too many tokens | 3 | 1175 | April 28, 2024 | |
| Do assistants count messages in the thread against the tokens limit? | 2 | 1781 | December 17, 2023 |