I did a fine-tune of a 70K words books. My initial expectation was to have the desired QA, and at that point I didn’t know any better. But this fine-tune showed me the limits of this approach. It just learned the style and stayed more or less within the corpus, but hallucinated a lot.
Then I split the book into sentences, worked my way through embeddings, and now I have a very decent QA system for the book, but for narrow questions. It is not as good for questions that need the context of the entire book.
I am confident that the future will bring the grail we want.