I am trying to code a tool to be used to generate “short” stories that will exceed the token limit. I have seen some interesting comments about summarizing the previous sections but I am having trouble making gpt-3 generate responses that easily can be joined together. Any suggestions about joining two generated sections or a better method for generating long responses would be greatly appreciated.
OpenAI recently published a recursive way to summarize novels. There may be a way to reverse the process. Summarizing Books with Human Feedback
In my experiments, it is difficult to get it to build off of summaries though. The hardest thing is generating an accurate enough summary to continue the story.
5 Likes
Hello @SecMovPuz, welcome to the OpenAI community!
If you want another suggestion, albeit less efficient than what @daveshapautomator suggestested, nothing stops you from summarizing each segment and structuring the prompt so that GPT-3 sees the summary and what to summarize next! I guess it just comes down to how comfortable you are with the API.
Feel free to reach out if you have any questions! We’re here to help 
2 Likes
This thread has made me want to revisit my novel-writing idea. Gonna spend today coding and if it produces good results I might seek out an investor.
Here’s the first attempt. It tends to get stuck on repeat.
Transgalactic flights used to be so boring, that is, until the Obelisk was found. The first hint of it was a sensor ping more than a decade ago, but the crew of that ship dismissed it as a blip. It would be impossible for an object to be several parsecs across. And since they were travelling at relativistic speeds in the intergalactic void, it was difficult to pin down exactly where it was.
But I found it again.
My name is June and I’m an explorer. My ship is called Daedalus and we’ve been travelling for twenty years. We were the first to find the Obelisk, and we’re the only ones who have visited it.
Now that I’m back in my home system, I can’t help but feel like something is missing. Like a piece of me is gone. I can’t shake the feeling that I’ve forgotten something important.
I’m going to have to go back.
The Obelisk is a diamond shaped object, about a thousand miles long and half as wide. It’s not made of any material we can recognize, but it does have a gravitational field. We’ve been able to get a few probes and drones close to it, but they’ve all been destroyed.
We’re not sure what the Obelisk is made of, but we think it’s organic. It has a magnetic field that seems to be alive, and we’ve seen some strange things happening at the poles. The only way we can describe them is like auroras, but they don’t move like any aurora we’ve ever seen. They’re almost like arms reaching out from the Obelisk.
The Obelisk has been sending out signals for years now, and we still haven’t figured out what they mean. We think it’s trying to communicate with us in some way, but we just don’t know how yet.
I’m going back to find out more about this thing, whatever it is. I have a feeling that if I do this right, I’ll be able to make my name as an explorer for years to come." June looked out the window and watched as her home grew closer and closer. It was a tiny speck in the distance, but it had been her home for twenty years. She couldn’t wait to see her family again, but she also felt like something was missing.
She tried to shake the feeling off, but it wouldn’t go away.
“You okay?” asked one of her crew mates from behind her.
“I’m fine,” June replied without turning around. “Just thinking.”
“About what?” he persisted, walking up beside her and looking out the window with her.
June sighed and turned to face him before answering his question: “The Obelisk.” “The Obelisk?” he asked, confused.
“Yes,” she replied. “It’s the reason I came out here in the first place.”
“I know that, but what about it?”
June turned back to look out the window again and said, “I think it’s alive.” “I think it’s alive,” she said, turning back to look out the window.
“What do you mean?” he asked, coming over to stand by her side and looking at it.
She shrugged. “When I was approaching it, before we landed on its surface, I could feel something calling me towards it.”
“Have you been getting any strange readings from the obelisk?” he asked.
June thought for a moment and then replied, “No.” “I think it’s alive,” she said, turning back to look out the window.
“What do you mean?” he asked, coming over to stand by her side and looking at it.
She shrugged. “When I was approaching it, before we landed on its surface, I could feel something calling me towards it.”
“Have you been getting any strange readings from the obelisk?” he asked.
June thought for a moment and then replied, “No.”
I just had to write the first couple lines as a seed. Using either DAVINCI or INSTRUCT it really cannot seem to integrate a summary. If there is a summary at the beginning, it just generates a wikipedia-like article as though it’s describing a book rather than writing the actual novel. It would probably require a fine-tuned model to be able to integrate a summary…
1 Like
Okay, got a little further but then the summary become too long:
Transgalactic flights used to be so boring, that is, until the Obelisk was found. The first hint of it was a sensor ping more than a decade ago, but the crew of that ship dismissed it as a blip. It would be impossible for an object to be several parsecs across. And since they were travelling at relativistic speeds in the intergalactic void, it was difficult to pin down exactly where it was.
But I found it again.
My name is June and I’m an explorer. My ship is called Daedalus and we’ve been travelling for twenty years. We were the first to find the Obelisk, and we’re the only ones who have visited it.
Now that I’m back in my home system, I can’t help but feel like something is missing. Like a piece of me is gone. I can’t shake the feeling that I’ve forgotten something important.
I’m going to have to go back.
The Obelisk is a diamond shaped object, about a thousand miles long and half as wide. It’s not made of any material we can recognize, but it does have a gravitational field. We’ve been able to get a few probes and drones close to it, but they’ve all been destroyed.
We’re not sure what the Obelisk is made of, but we think it’s organic. It has a magnetic field that seems to be alive, and we’ve seen some strange things happening at the poles. The only way we can describe them is like auroras, but they don’t move like any aurora we’ve ever seen. They’re almost like arms reaching out from the Obelisk.
The Obelisk has been sending out signals for years now, and we still haven’t figured out what they mean. We think it’s trying to communicate with us in some way, but we just don’t know how yet.
I’m going back to find out more about this thing, whatever it is. I have a feeling that if I do this right, I’ll be able to make my name as an explorer for years to come."
June and her crew set out in their ship, Daedalus, to go back to the Obelisk. They thought it would be difficult, but when they reached it they found that many people had already come and gone before them. It didn’t seem like there was anything worth exploring anymore.One day while she was exploring the Obelisk on her own (she wasn’t sure why) she came upon a doorway made of light. Upon entering this door, she discovered an entirely different world with talking animals and plants which sang when you stepped on them. She started running around trying to explore more of this place until she heard a voice behind her say “I told you not to run.”
All of the sudden there were three tall creatures that looked like trees standing in front of her. One said “We are sorry for your loss.” Another added “You have been here before.”
June starts to realize that this place is like her home. She has been here before, but she doesn’t remember it. It’s like the door unlocked something in her head and all of a sudden she remembers what happened before. She remembers walking through the doorway made of light with her crew, but they don’t know what’s happening because they can’t see anything on the other side except for June who seems to be floating in nothingness. They start running after her calling out “June! June!” But then they disappear and she wakes up back in reality on Earth, no longer an explorer but a woman whose children were taken from her long ago by the government.
June starts to realize that this place is like her home. She has been here before, but she doesn’t remember it. It’s like the door unlocked something in her head and all of a sudden she remembers what happened before. She remembers walking through the doorway made of light with her crew, but they don’t know what’s happening because they can’t see anything on the other side except for June who seems to be floating in nothingness. They start running after her calling out “June! June!” But then they disappear and she wakes up back in reality on Earth, no longer an explorer but a woman whose children were taken from her long ago by the government.
When the crew saw her disappear, they started to run after her, but soon realized that she woke up back on Earth. The last thing she remembers before waking up is running through a bright doorway with her crew. They start trying to find out what happened and find June’s original journal entry about entering this world previously. They travel through the Obelisk again following the same steps as before and eventually reach the doorway made of light. However, when they enter this world, it doesn’t look anything like it did for June before. It looks more sinister–darker colors, less nature-like. Almost all of the creatures are dead or dying and there are people everywhere who have been tortured horribly by some unknown force. The crew comes across one person who remembers their previous visit to this place but has no idea why it changed so drastically in just twenty years’ time. One of them says “Maybe we shouldn’t go any farther.” The other two reply “We can’t stop now.” But then they hear somebody say “I told you not to come back!” And three trees stand in front of them again saying “We are sorry for your loss” and “You’ve been here before.”
The crew tries to go on despite warnings, but soon hears somebody yell “I told you not to come back!”, causing three trees to stand in front of them again saying “We are sorry for your loss.” One says “You have been here before,” while the other two add “We can’t stop now.” The three tall creatures say that they will never be able to leave the Obelisk and they ask June if she wants a second chance at life. She replies yes and gets transported back into reality, this time with her family.
June and her crew were exploring the Obelisk and had found a doorway made of light. They soon found themselves in a strange, new world with talking animals and plants that sing when you step on them. June was running around trying to explore this place until she heard a voice behind her say “I told you not to run.” All of the sudden there were three tall creatures standing in front of her who looked like trees. One says “We are sorry for your loss,” while the other two add “You’ve been here before.” June starts to realize that this place is like her home. She remembers walking through the doorway made of light with her crew, but they don’t know what’s happening because they can’t see anything on the other side except for June who seems to be floating in nothingness. They start running after her calling out “June! June!” But then they disappear and she wakes up back in reality on Earth, no longer an explorer but a woman whose children were taken from her long ago by the government.
June gasped as she remembered the creature’s words. “You’ve been here before.” She had always thought that it was just a coincidence, but now her memories were filling in the gaps.She was with her crew on their first exploration of the Obelisk when they came across a doorway made of light. They went through and found themselves in an entirely different world with talking animals and plants that sing when you step on them. June started running around trying to explore more of this place until she heard a voice behind her say “I told you not to run.” All of the sudden there were three tall creatures standing in front of her who looked like trees. One says “We are sorry for your loss,” while the other two add “You’ve been here before.” June starts to realize that this place is like her home, she remembers walking through the doorway made of light with her crew, but they don’t know what’s happening because they can’t see anything on the other side except for June who seems to be floating in nothingness. They start running after him calling out “June! June!” But then they disappeared and she wakes up back in reality on Earth, no longer an explorer but a woman whose children were taken from long ago by the government.
June woke up in a lab, surrounded by white walls and a buzzing sound. One of the doctors walks over to her, checking her pulse. He says “You’re going to be fine.” She can’t remember anything about what had happened, but she knows it was something bad because as soon as she wakes up she starts crying hysterically. The doctor tells her that they found her freezing to death on the side of a road, and now they’re trying to figure out how long ago it happened. June does not know how much time has passed since then based on how long it takes for dreams to fade away after waking up from them. She remembers the events of before like they just happened yesterday but at the same time feels like she has been there for years. Suddenly another doctor enters the room and starts asking questions about who had taken care of her while she was gone. She doesn’t know what he means so he explains that apparently somebody had taken care of her for five years while she was asleep - feeding her, changing clothes, everything! Eventually though June realizes that this person is probably also one of those creatures from before with their warning not to come back - which gives even more credence to their words than before!
June woke up in a lab, surrounded by white walls and a buzzing sound. One of the doctors walks over to her, checking her pulse. He says “You’re going to be fine.” She can’t remember anything about what had happened, but she knows it was something bad because as soon as she wakes up she starts crying hysterically. The doctor tells her that they found her freezing to death on the side of a road, and now they’re trying to figure out how long ago it happened. June does not know how much time has passed since then based on how long it takes for dreams to fade away after waking up from them. She remembers the events of before like they just happened yesterday but at the same time feels like she has been there for years. Suddenly another doctor enters the room and starts asking questions about who had taken care of her while she was gone. She doesn’t know what he means so he explains that apparently somebody had taken care of her for five years while she was asleep - feeding her, changing clothes, everything! Eventually though June realizes that this person is probably also one of those creatures from before with their warning not to come back - which gives even more credence to their words than before!The last thing June remembered before waking up was running through a bright doorway with all three crew members following closely behind when suddenly there were three tall beings in front of them again saying “We are sorry for your loss,” and “You’ve been here before,” while looking at each other knowingly as if having an unspoken conversation between themselves which only included themselves- no matter where someone stood or where they pointed their flashlight beam at them you couldn’t see any trace or outline apart from these three figures standing in front- blocking their way entirely until two more trees stand up saying “You have been here before” while the third adds “We can’t stop now.” All three creatures say that they will never be able to leave this spot until someone grants them a
It still tends to be highly repetitive. I turned up the temperature and penalties, which helped a bit. Part of the problem is that the human brain can hold a ton of details with very sparse representations AND our brains continually engage with memory as we read and write. I suspect, in order to write a novel with GPT-3, we will need to take a “one sentence at a time” approach, where we spend a lot of time recalling all relevant details from the entire story and distill it all into a massive prompt and use that prompt to generate a single sentence. The problem here is that GPT-3 is trained on continuous text, so this approach will absolutely require a fine-tuned model.
Update: It only cost $2.70 for all the experiments to lead up to this point. What you’re looking at here only cost about $0.80 to generate, including the prompts, give or take about 20 cents.
I’ll be mulling what it ought to take in terms of fine-tuning models to actually write a whole novel but here’s what I’ve got so far:
- Crazy good summarizers. The hardest part is getting all the relevant details as concisely as possible.
- I think you might also need “meta-summarizers” - as in something that will further compress the summaries. Maybe in the form of bullet points in order to save tokens.
- A fine-tuned model that takes in the summary, the last few lines, and generates the next sentence or two.
- An entirely different set of revision/editing models…
It occurs to me that several tiers of summary might work well in order to achieve sparseness. that is to say, the further back you zoom out, the less resolution you need. So you could start with a “summary of the whole book so far”, and then a “summary of the last chapter”, finally followed by “summary of this chapter so far”. Something along those lines. Either way, a good fine-tuned model that accepts summaries and a few lines to cue off of should be able to handle that.
Furthermore, writing novels is an iterative process of sequential refinement. Even if the first draft is nearly unreadable, we could then use a different set of models and tools to automatically refine the story. As an author myself, I will say that revising, editing, and refining uses a completely different set of cognitive abilities. Fellow writers will often talk about how starkly different the mental headspace is. GPT-3 excels at filling up a blank page, which is the hardest part for some writers.
2 Likes
Thanks for the suggestions so far. They are pretty insightful. I will have to get coding and training!
Here’s the repo I made yesterday. It’s only got 1 python script so it should be easy enough to understand. The biggest problem I ran into was that summary length got too big, but I’m sure with some work that problem could be overcome. I hope these ideas help.
2 Likes
If you haven’t already done so, I would recommend taking an iterative approach to summarizing the larger content. Use an NLP library (Spacy, Gensim, or NLTK) then break your story down into paragraphs. Once in paragraphs, separate them by sentences. Use some summarization method on the paragraph itself, then on each sentence individually. Once you have a summarization of the sentences, compare these to the paragraph summarization. Did you like the result? If so, put the paragraph summarization and the sentence(s) summarization into yet another summarization. Did this maintain your data’s integrity? If not, another approach would be to replace keywords with a relatively close synonym. If you use Word2Vec for comparing synonyms, you could technically say the same thing but using different words. I realize this was a ton of suggestions. Feel free to use one of them to assist you in summarizing your data. Cheers!
3 Likes
Is this recommendation based on experience or just a personal hypothesis? Anyways, conventional NLP approaches either don’t work or aren’t necessary with GPT-3.
The recommendation is based on both. I have been trying to create new context whilst keeping the data’s integrity. However, I don’t have experience concerning your precise model (listed previously). This link provides the code for using spacy-wordnet. This one might be useful for summarization. If you don’t mind, I will send you a message because I would like to discuss your thoughts with regards to the latter part of your message.