How long is it optimal to keep the message for the chat history

The token size of requests in API use is important for costs. I forward the previous messages with a list each time so that he remembers the chat history. How many messages do you think keeping is good for temporary memory?

You could start by summarizing older longer responses or prompts. This way the information slowly vanishes.

1500 words works for me. I summarize it into 500 words while maintaining the first system and user message.


How? If you use embeddings, you can stuff highly correlated previous conversations into the chat history.

This enables the bot to switch topics quickly and sound like “a good old friend” that has a deep history with the user.