I want to limit the input tokens of assistant, because in the new model gpt-4-1106-preview input could be up to 120k tokens which means if my message history grows to 120k tokens I would pay $1.2 per message…

As of now, you cannot set the token limit for an Assistant beyond choosing a model with a lower context window. We have received a lot of feedback that this would be really useful so the team is looking into it. Today, the Assistant will try to keep as many messages in context as it can and naively…

How to limit input tokens of assistant?

API

fra_ab November 30, 2023, 2:57pm 2

Topic		Replies	Views
How many tokens is the size of the context window in Open AI Assistant? API	5	2620	April 8, 2024
OpenAI Assistant maximum token per Thread API gpt-4-turbo	11	10580	May 28, 2024
Request: make it possible to specify the upper limit of history Feedback assistants-api	11	1013	July 10, 2024
Assistant API Max input context size API	5	1942	April 16, 2024
Token consumption: Prompt tokens exponentially increase when using Threads (Assistants) API assistants-api	8	375	September 5, 2024

How to limit input tokens of assistant?

Related topics