A question about the context. May I ask everyone

[419649693]

Due to the need to provide all previous chat records in the implementation context, this will result in significant token usage. So I would like to ask everyone, is there any way to implement context while reducing tokens?
It is best to use techniques to reduce the length of the context.
Or let’s talk about your opinions.

[damc4]

I’ve just described something that answers your question here:

Question about token limit differences in API vs Chat

Because the way large language models work is that they have context window, so the prompt has a limit of how long it can be. That’s why API tells you that you are over the limit. However, there are ways to deal with that problem that are implemented on the website, so that’s why you don’t get that problem on the website. The well-knows methods to deal with that are as follows: Sliding context window - if you have a chat with more than 4000 tokens, you send only the last 4000 tokens to API, s…

Please have a look.

You can use those techniques to lower the token usage as well.

[Foxabilo]

You need to pass the context you think relevant to the model to be processed. There is no “old memory” within the GPT Models, so you need to send everything, or you can filter your context in some intelligent way. That filtering usually requires some other form of AI… so in the end you need to process your data and that takes tokens.