What I really want to do is a tool like they made in Best Seller Code…
You upload your novel and it tells you whether it will sell or not… Or edits it to make it a best seller…
1 Like
worth noting that Echoes of Oblivion is no edits. The output is a single forward only pass. If you even made just one additional editing pass I think you could dramatically improve the quality of the final result.
2 Likes
Yeah, what I noticed is that if I edit as I go it results in better output because it’s basing new content off what it’s written already… so if you correct it early, it’s smoother all around…
I know what you mean about LLMs only wanting to write about 1200 words at a time. o1 is willing to push it, one time I got o1-preview to output 4500 words in one go, but it was reluctant and noted that its guidelines suggest a max output of 500 words.
In the novella project I’m working on, we’ve simply subdivided the chapter and fed in the previous chunks as part of the context window when writing part 2 or 3. That’s worked well, since then we’re not fighting its impulse to write in 1200 word blocks. Similar to the outline you provided, we don’t at this point give the system the entirety of the novella as context, just a detailed plan of where the story has been and where it’s going. That fits into your comment about breaking up the novel and writing parts separately - that’s working well for us!
1 Like
I’m assuming your outline has to drill into the specific scenes that should playout within a chapter then… Do you see cohesion issues when stitching the subparts of a chapter together? Or does the model naturally pick up the scene where it left off? I’m assuming you still have to stop at logical break points within the chapter.
1 Like
With technical writing it’s a bit more difficult to break mid section (chapter.) The audience we’re targeting are product/project managers and we’d be doing good to get them to think about the sections they want in a given document. They’re not going to want to drop to the sub-section level.
With Echoes of Oblivion I had o1 generate a detailed summary of each chapter and told it to include details about characters, plot points, story arcs etc… I include the last chapter in full and then a summary of the chapters before that along with the outline. I saw around a 50% compression using this technique but in retrospect I wouldn’t recommend it…
The issue is that, as the story progresses you’ll notice formatting from the summaries start to creep into the output. These models are just pattern matchers and they naturally want to mimic the patterns they’re shown in their output. So when you reach a tipping point that you have more cliff notes then content, any new content starts to reflect the structure of the cliff notes.
I suspect that if you’re willing to spend the tokens, the best approach is to show the model the outline plus everything it’s written so far. The outline helps it know where it’s going so you could limit this to just “what’s coming” and seeing the full content of everything it’s written will give you better cohesion across chapters. A full novella is under 64k tokens so you can easily fit it plus the outline in a context window.
1 Like
Could I share your science fiction novel in my forum?
Of course. It’s open source. I’ll generate a v2 after I finish my tweaks to the generation algorithm. I think I’ve got the cohesion issues ironed out.
1 Like
o1-mini ChatGPT is definitely not the model for this.
Complete outline and instructions given, to only get 60% to where a “continue” would happen from max-tokens (but that’s not much different than other models’ need to wrap up). However, it just goes off the rails into AI cliche, and then the premature closure after ambiguity and joyfulness is bizarre “about the author” that has absolutely nothing to do with the prompt.
ChatGPT’s model might have seen that there is no need for reasoning, with one step shown, just turning over output to the final AI.
A dump out of some footnote of nonsense
…In the grand tapestry of the universe, our stories might seem like mere threads, but together, they weave a narrative of resilience, exploration, and the relentless pursuit of understanding. The Draco Tavern stood as a microcosm of that larger journey, a place where the extraordinary met the ordinary, and where, amidst the chaos and the calm, the true essence of our collective existence was revealed.
As I prepared to leave, the first light of Neo-Orion’s moons began to filter through the windows, casting a serene glow over the tavern. I took one last look around, committing the scene to memory—the friends who stood beside me, the remnants of chaos now subdued, and the silent promise that, come what may, the Draco Tavern would continue to be a haven for those seeking both solace and adventure in the endless expanse of space.
With that thought, I stepped out into the cool night, the echoes of the evening’s events lingering like the fading notes of a well-played melody. The Draco Tavern awaited my return, ready to welcome the next chapter in its storied existence.
Rick Jansen is a freelance reporter and chronicler of interstellar events, often found at the heart of Neo-Orion’s most intriguing locales. His tales capture the essence of life on the fringes of the galaxy, where every night holds the promise of the extraordinary.
For some reason, it reminds me of Mixtral 46B, where you get little imagination beyond what you input.
1 Like
Earlier this year, I found Goliath to be quite creative for fiction but it was “clunky” and slow…
I’ve not tried any models from the last few months, but I might be back to writing fiction soon. Someone should do a write-up on the creativity of all the models when it comes to fiction…
Sharing my progress on Echoes of Oblivion v2:
This is just the prologue and the first 3 chapters but I wanted to share the progress because it’s really good… I mean really good…
I fixed the cohesion issues but beyond that I had it define its own writing style based on the great science fiction writers. I also had it generate the first few chapters, think about what was working and not working, re-work its outline to include detailed plot points, and then we started over. What’s being generated now is night and day better then v1.
Get beyond 20k tokens and o1 really starts to struggle. It starts rushing through plot points and it essentially blurring chapters together. I’ve seen this with gpt-4o as well. It’s definitely improved with o1 but it’s still there.
I’m trying everything I can think of to get o1 to just slow down it’s thinking… That’s what its supposed to be good at right? It’s just a difficult ask…
I spent several hours (and about $50) working on this last night and o1s reasoning just starts breaking down when the input context gets above 20k tokens. It’s subtle but the model starts getting confused about facts and struggles to follow instructions. It’s probably task specific and it’s not like it doesn’t try to do what it’s asked. There’s just a clear loss of focus.
Every model I’ve tested has the same issue but it’s more disappointing with o1 because it’s just so damn good up to 20k. 
3 Likes
For long-form fiction, you’ll need to generate text based on the underlying meaning of the novel (the subtext). If you can find a framework that models what stories mean rather than what they say, then you don’t have to worry about token length or context windows.
We’ve found o1 to be exceedingly great at helping unravel that underlying meaning, and for capturing the author’s intent. Our latest blog post covers the difference between o1-preview and o1-mini in finding this subtext (i.e., the difference between Paramount+ and A24):
From Popcorn Flicks to Profound Narratives: How Muse Finds the Heart of Your Story
1 Like
The underlying goal of @stevenic work, as best as I understand it, is to examine the limits of the breaking point of the o1 model. How much context (instructions and facts) can it keep “in mind” as it continues to generate the next word. ofc the analogy of “keeping in mind” is not completely appropriate; but best that I could think about.
All by itself, I don’t believe that o1 is capable of unraveling the meaning and implications of what it is uttering one word at a time; at the same time that it is uttering it.
1 Like
What AI platform are you using? I have a fiction I’ve been working on for quite some time. I have 10 chapters done… And I’m winding down to the end, but I’m just trying to work out a couple scenarios I’d like help with it. Is that something you think I could use AI with?
Yeah… I’m intentionally trying to push o1 to its limits. I do have a real-world writing task I intend to use o1 for but it’s for technical writing. I was just exploring fiction writing because it lets me poke at the models raw instruction following capabilities as the context window grows in size. It seems rock solid up to around 12k-14k tokens and then starts to slightly drop off. At 20k and beyond it really struggles to follow instructions.
I read your post and would agree with the observation that o1-preview seems really strong at story telling. I briefly gave o1-mini a try but it was clear that it’s not in the same league as o1-preview for this task.
I’m curious about a couple of thing and feel free to share what you’re comfortable sharing. I’m assuming you’re creating some form of outline that you have the model writing to and my first question is do you pass just your outline in when you generate a chapter?
I was passing in both an outline and the story text that had already been generated since theoretically you can fit an entire novella in less than 64k tokens. The issue I was seeing is that the model stops following the outline when the input context gets above 20k tokens. It basically stops following instructions.
What it generates at that point isn’t horrible it just feels rushed because it takes plot points slated for future chapters and starts mixing them into the story early.
If you’re not seeing this I’d have to assume it’s because you’re taking a different approach that lets you keep your context window length down.
Don’t mind at all sharing–
–if you just think of it in terms of how an author would go about writing a story, it’s the same thing.
Assuming you could break authors into two broad categories - those who write to an outline, and those who discover it (and by authors I mean both the human-type, and the LLM-type) the first group would work the way you suggest: find the unique outline, and then use the LLM to think through and logic/emote the subtext underneath (what the story means). This would be the Subtext pass.
Then, you take that generated outline/treatment and use that as the basis for writing individual “scenes”. Since the scenes are already tied together through the subtext pass, you can allow all the generative/brainstorming ingenuity you want without worry of the story going “off the rails”. This would be the Storytelling pass.
This works great for authors who already know what they want to say as they can quickly generate/think through the Subtext, and then start the actual “writing” (Storytelling).
For those who prefer to write their way into a story, the overall framework/strategy is the same, you just have to find a way to the Subtext while working through the Storytelling. With this group of Authors, you would first do a chunk of Storytelling, appreciate the meaning of what is being written and find the bits of Subtext underneath. You could then suggest alternate “paths” for their Storytelling based on what you know about the Subtext, or simply continue to monitor as they write (and they could be the authors themselves, or the LLM).
As the Subtext of a narrative is contingent on the relationship of all the parts, you might not fully get to the meaning until the end (or maybe 3/4 of the way through), and at that time you might have to re-contextualize/re-write some of the Storytelling to fit the Subtext.
Again, this is no different than the way storytellers have worked for centuries. Many don’t realize the subtext of their work, many refuse to acknowledge/don’t want to know, yet it’s always there. In essence, you would be modeling the subconscious desire to express/find meaning within the text of a story.
The trouble lies in trying to train off the text of a work without acknowledging that there is something deeper going on. Then you’re just RL’ing the style and storytelling “talent” of the individual author.
For instance, you could train off of the Storytelling of Shakespeare’s plays, like say for instance “Romeo and Juliet”, and you could prompt “write me the same story that takes place in the 1950s in the Upper West Side of New York City”, but it’s highly unlikely that you’ll end up with “West Side Story”, without totally losing sight of what it’s all about.
Now, if you instead trained off of the Subtext of “Romeo and Juliet” and then set about writing individual scenes and sequences that related back to that…now you’re getting much closer to a narrative that will emotionally connect with them and have that same sort of feeling you get after experiencing a great story.
2 Likes