I think the history is mostly based on summaries, rather than embedding similarity.
Also, the models do surprisingly well if you ONLY remember the user questions, not the answers.
But, in general, if you want very long conversations, you have to use heuristics for which previous questions and answers are really relevant. For example, if you iterate on some code, you only need to remember/re-prompt the latest version of the code, not the previous iterations. Exactly what this means, is application specific.