I’ve been experimenting on a python script that will allow a user to write a novel with gpt4. Currently, it generates a synopsis and chapters that then initiate the pages to be written from the synopsis / chapter title. I’m running into an issue of token size limitations for writing out pages of the novel. I’ve experimented with breaking the pages down into chunks, but find when the next chunk is being written it usually just starts repeating itself over and over. Is there a way to have GPT 4 “memorize” what it’s previously written to continue its story? I assume if I added everything that was previously written it would use up all the tokens for the prompt.
The AI model has no memory, as you likely understand.
You provide input, it produces an output.
It is rather, then, about the technique that you would use to provide continuity in few words, so that a single run of the model can produce the continuance text you want.
First, you can remove the max_tokens parameter, so that there is no artificial limit or miscalculation that prevents you from getting the maximum output before truncation.
Then, you simply could press “continue” - but that would be impossible if you have your 8192 tokens already from that last call in the form of outline, overall plot, chapter summary, and now the new writing that would take what you sent up to the limit. You need to figure out what can be compressed again before you can re-send.
One idea would be to send your chapter so far for summarization to gpt-3.5-turbo-16k. It can understand the 15000 tokens of multiple runs of chapter formation, and be able to compress the events of the chapter so far down to its pretrained “summary” length of around 500 tokens. Then you can continue with the same overview and other inputs, but at least then the summary of what has been written so far in achieving the goal of the chapter from the chapter outline.
1 Like
Appreciate the tip! I will see if I can add summarizations and keep the general theme for the chapter/synopsis and play with this method.
If you search around the forums, you’ll find some good gems. Good luck on your coding journey!
2 Likes
Much appreciated, I’ll browse the forums a bit more in-depth. If you have any references or links please feel free to share! Take care.