Assistants API context window?

Nope. It’s mind-boggling why this isn’t a feature.

The only solution feels hacky and it’s basically to count the tokens yourself by retrieving all the messages (waste), counting the tokens (waste), summarizing the conversation (waste), destroying the thread and re-creating it with the summary prefixed in the user message or instructions.

You cannot even truncate it yourself. Adding an assistant message isn’t permitted. So realistically the solution is to either drain your bank account or roll back to an inferior model that still at times will max your tokens by falling into an infinite loop.

100%. I love the concept of Assistants and am building my tools to use them as well in hope that they are improved. After the ousting though I’m not even sure how long it will be until it’s addressed.

6 Likes