You would ideally have your own database. However all that’s needed is your own trimming function (usually done by summarizing the conversation)
You can use any type of database - even local files, however with scaling & design philosophy in mind I’d say it’s a good idea to get comfortable with vector databases. I would recommend using Pinecone. Not only is it free, it’s lightning fast, and they are constantly implementing new features to adapt perfectly for more use cases. It’s also very nice if you’d like to perform analytics on the sessions based on their semantic relevance.
In this case, I’d set the conversation session ID as the namespace and use that for retrieval.