Chat with long documents (eg books) with long term memory

Hi, I plan to work on a personal project to chat with documents.

I know chatting with documents is a common use case.

But I could not find enough resources/code that handle following cases: with very long documents like books (so that I can upload document beyond GPT4 token context)
2.have long term memory (more than what GPT4 can provide)

I want to chat with long documents and have memory of all the previous conversation, just like ChatGPT UI provide but using it beyond token lenght limitations.

I know that I need chunking, embedding, use vector databases, and probably using Langchain would be great.

Most of the resources on Github lack one of the features?

Can you please share any resources that can help me start with the right foot so that I can avoid figuring out what already has been there.

I can not imagine that has not been done before :slight_smile:

Thank you.

1 Like