I want to limit the input tokens of assistant, because in the new model gpt-4-1106-preview input could be up to 120k tokens which means if my message history grows to 120k tokens I would pay $1.2 per message…

As of now, you cannot set the token limit for an Assistant beyond choosing a model with a lower context window. We have received a lot of feedback that this would be really useful so the team is looking into it. Today, the Assistant will try to keep as many messages in context as it can and naively…

[image] How to limit the number of messages or tokens that are persisted in a thread to maintain context in Open AI Assistants? API I read the documentation and it says there is no limit, the only limits is the model’s context. But I want to control how many messages or to…

That’s fine… I can indeed limit the number of messages I’m sending to the model, but would be nice if you can share best practices for such a solution… For example does it matter if oldest message in history is assistant’s message or user… Or any recommendations would be much appreciated! Thanks in…

Hi logan Thanks for your clarification , speaking of today’s date is there still no way implemented to limit the tokens ?

any updates on this? Ik you guys are swamped but this would be really big if we can control the tokens.

How to limit input tokens of assistant?

API

fra_ab November 30, 2023, 2:57pm 2

Topic		Replies	Views
How many tokens is the size of the context window in Open AI Assistant? API	5	3819	April 8, 2024
OpenAI Assistant maximum token per Thread API gpt-4-turbo	11	11999	May 28, 2024
Request: make it possible to specify the upper limit of history Feedback assistants-api	11	1312	July 10, 2024
Assistant API Max input context size API	5	2204	April 16, 2024
Token consumption: Prompt tokens exponentially increase when using Threads (Assistants) API assistants-api	8	1034	September 5, 2024

How to limit input tokens of assistant?

Related topics