Hello everybody!
I read all the related topics however haven’t found the answer for a thoughts I have.
I want to train the model with the content of the book and it’s obvious for me that I should use RAG and embeddings. It’s great for retrieving information about details of the book content (the details included in certain part of the book). However except that, I also need my model to know the full context of the book (i.e. about some transformation of main character throughout whole book).
I tried fine-tuning, however there were no satisfying results, do you recommend some other approach for that?
I will be very appreciated for your help, thank you in advance!
1 Like
Welcome, @mateusz.jankowski.92, to the best AI developer community online!
I would recommend RAG for knowledge more-so than fine-tuning. The latter (fine-tuning) is more for style rather than knowledge acquisition, if that makes sense?
What you would likely do it compare the user’s input to your vector stores to find the information needed. You said that is working, but you want to fine-tune as well?
1 Like
Hi @PaulBellow, thank you for your quick response! So what I would like to achieve is flexibility:
- I would like my model to give correct answers about particular events, aspects, situations of the book (for this purpose RAG is great - it will provide great and precise answers)
- I would like my model to be able to answer the questions about general content of the whole book (as I imagine, for this purpose RAG will not be the best solution because picking the best part of the book will not allow the model to answer such kind of questions)
And second bullet is my main concern.
1 Like
My first thought would be to break the book into sections and try to summarize them to get an idea for the overall book. Otherwise you need to send the entire book every turn, which isn’t optimal! 
Paging any fine-tune experts around, but I don’t think it will work well.
I second what Paul says, its a limitation of context windows that you may be facing, ChatGPT cannot continuously have everything in its context window and drops things it (including what openai )deems unimportant, its a physical limitation constraint of processing high data volumes within an acceptable time frame, something openai and all ai suffer from atm, really there needs to be another way to allow a neural net to have a continously forming and degrading memory that prioritizes certain memories but that to my understanding is the current fronteir challenge of ai
1 Like
@PaulBellow @liam.dxaviergs thank you guys for your answers.
So as I understood, what I would like to achieve (to teach the model whole context of the book) is in your opinion impossible? If so, then what you recommend is to use RAG and if user asks the question about whole book in general, then I should send it to openai api together with summary of whole book?
Would be appreciated for the confirmation that I understood you guys correctly 
PS Actually I have one more question about how model works - you say it’s impossible to teach the model content of whole book, however chatgpt knows content of some books. How is this possible then, is it possible during model training but impossible during fine-tuning?
fact check this cause im still learning things myself and can get things wrong. but from my understanding its trained as in its inclination towards certain patterns that represent the data like a designed bias based on the provided prompt. a neural net like an LLM has a limited input window and a limited output window, so it cant physically read the entire book only the patterns that result from algorithms that reduce the text to a pattern (because the input matrix would need to be as large as the text volume itself and this is computationally impractical). to clarify about the patterns, patterns could mean patterns of themes, patterns of words and sentiments, patterns of emotional aspects based of matching patterns of predifine emotional reference data. so when chatgpt “reads a book” its reducing it to a specific conceptual pattern via its algorithms then rebuilds aspects of the book from that via the probablisitic creation of the next most likely word (word: a series of tokens that align with each other to form a word) to answer the prompt. im glossing over alooooooooottt of code and very specific nuanced aspects of how nueral nets and LLMs work and very much over-generalising, and im still learnign some myself so take whati say with a decent level of skeptacism. but essentially, algorithms that reduce training data to a pattern, biases that change the patterns to a more logical and coherent pattern based of predefined context, pattern injection to nueral net input via a matrix, the hidden layers of the nueral net do some magic (learns patterns and outputs data based on probablistic mathematics and some other aspects, big oversimplification), the neural net outputs a value and this is added to the next portion of a response ( a token ), then the whole thing cycles again for each token thereafter until it forms the whole response. this is just one way a neural net works, and the GPT in chat GPT stands for general pre-trained transformer, as in the patterns of itsreference data are reduced to patterns that can be put into the nueral net or “pre-trained” which is then put into the “transformer” which is the neural net setup of probablistic token prediction.
so to answer you question yes, with a GPT transformer you will need a RAG
to further analgyse, think of it like a tree, where all of the raw data is being fed into the tree via roots (the pattern algorithms) that then transform the data into usable fluids and nutrients and passed through the input (the trunk), then gets split into the hidden layers (the braches) and then the output layers (the leaves), and you see the leaves (or repsonses), you cuold also analygyse your prompt as the light for photosynthesis.
for a visual representation heres something analygus, makes me think of norse mythology and the tree of creation “yggrasil”
2 Likes
