Flesch Reading Ease Score Implemented with ChatGPT API

Normally getting reading level isn’t so hard with a short prompt, but I’m also controlling other things like tense, tone, etc. that made it a bit more difficult… along with the new range for temperature on chat-gpt-turbo…

(New ChatGPT API Temperature Note: On text-davinci-003, I routinely use temps of around .80, but with the new range, I settled around 1.2, but I still might change it after more testing…)

I knew I couldn’t get fine-tune control with settings from 1 to 100 like normal Flesch reading ease scores, so I started with trying for 10… I couldn’t get that much control over that many, so I dropped down to five…

  • Very Easy to Read (Flesch 90 to 100)
  • Easy to Read (Flesch 70 - 90)
  • Normal / Baseline (Flesch 50 - 70)
  • Hard to Read (Flesch 30 - 50)
  • Very Hard to Read (Flesch 1 - 30)

Each is about 20 points on the Flesch scale…

Here’s where I’m at currently with an example for each setting…

Very Easy to Read

You step into the cavern. You see a statue. It’s huge and carved. It’s a fierce dragon.

Your heart beats fast. You start to walk closer. You hear a strange sound. You stop in your tracks.

You turn around fast. You see the dragon move. It’s alive! It’s alive! You scream in terror.

The dragon looks at you. It’s eyes are glowing red. It breathes out hot air. It’s about to attack.

You must think quickly. You prepare to defend. You pull out your sword. You’re ready for battle.

The dragon charges fast. You move to dodge him. You strike with your sword. You defeat the dragon!

You breathe a huge sigh. You survived the battle. You feel so heroic. You defeated the beast!

Flesch Reading Ease Score: 100

Easy to Read

As you enter the cavern, your breath catches in your throat. In the dim light, you see a towering statue, its smooth stone face carved into a snarl. You know it’s the final guardian, the one you’ve been searching for. You steady yourself, heart pounding as you raise your weapon.

The silence is deafening as you move to engage the statue. In that moment, there’s nothing but the pounding of your blood in your ears and the sharp clang of metal on stone. It’s a fierce and deadly battle, both of you equally determined to emerge victorious.

Then, you strike the final blow, your weapon clattering to the ground as the statue crumbles into dust. You sink to your knees, gasping for air, victorious. As the dust settles, you know. Truly know: you are the chosen one, and nothing can stand in your way.

Flesch Reading Ease Score: 83.6

Normal / Baseline

You step hesitantly into the cavern, the damp and musky air thick with the stench of decay. Your hand grips tightly to the hilt of your sword as you scan the underground space for any presence of danger.

Your eyes are first drawn to a statue in the center of the room, its shimmery surface glimmering eerily in the dull light. It’s impossible to tell whether it was carved as a means of worship or a warning to those who tread within this space.

As you broaden your scan above the statue, you suddenly become aware of the hissing of a great beast, and your heart rate escalates immediately. Its eyes lock on to yours and immediately you know that only one of you is leaving this place alive.

You sense that this strange, glowing statue might be the cause of the turmoil and urgency in the dungeon scuffling behind you. You want more than anything to run in the opposite direction, but doing so assures certain death at the talons of the dragon before you.

You grit your teeth and move to hide behind the statue, blinded by the light. Courage fills you as you prepare to face the dragon. You strike with all the violence you have, and The screeching dragon cowers in fear, watching the death of the statue with which it seemed bound.

It becomes clear that slaying the creature it was connected to the great ornately carved statue dead-ended several deeper clues that would afford you escape. Tension wanes and replaces completely with utter disconnect you recall a puzzle further in the mountain that needed a decent band of energy this dragon had provided at the touch of Kir vi the enchanted long sword . Heavy weight falls over you as mystery and trepidation substitute into suspense and thoughts.

Flesch Reading Ease Score: 66.5

Hard to Read

You enter the subterranean realm and are confronted with a magnificent statue bestowed with a magical essence that radiates through its serpentine furrow. Its effulgent brilliance envelops the entire chamber in a warm, ethereal glow. As you approach the marvelous entity, it becomes clear that this is not mere artwork – this is an ensouled creature – incredibly ancient and powerful, bound within a lifeless shell. The majestic form towers over you, and you cannot resist the force beckoning you to approach. Suddenly, a sudden stir quakes within the statue as if it were awakening from a lengthy slumber into a newfound clarity of thought. Then, in one fell, lightning-swift motion, its eyes spring open, harsh, incandescent beams project outward like flames, and bolts of scorching magic arise, conjured before you, and you realize too late that this was but a prison for some greater evil lying dormant from ages past. And the colossus now charged to act expresses retribution at the interlopers diminishing its eternal peace with frenzied fury that could further rip through and scar reality for an unceasing span.

Flesch Reading Ease Score: 36.9

Very Hard to Read

You find yourself within the cavern, encircled by intransparent walls that seemingly constrain the cold unknowable depths of the earth. The sound of your steady treading reverberates within the carven formation like an impious ritual, as if rhythmic notes possessed a sacrality unheard of in mundane spaces. It is quiet, the brevity of life, an annihilating silence impartial at every command, constantly looming adjacent, amplified and enforced by the mineral essence of the elevated spatial emptiness that buries the remnants of origin, destroying the untouched landscape. Adroit moss braids savor every sort of fissure achievable, carving a splendid kaleidoscopic imagery interjected with milky marble detailing. And amidst the fascinatingly intricate pattern emerges a monument, immense and statuesque - a looming visage, a refutation on shyness incarnate of body curled gently alighted onto dais soft from obsidian. The colossus sprawled, the slightest chiseled flare offered provocation, wild gaze unblinkingly entrenched, her stone figure indecisively reflecting symbol language of the beyond- comprehendable. But just then, something strange happens. Indecipherable to the rational mind, a disquieting moment seizes you and an arresting queasiness erupts, your thought-chain stuttered as hazily reimagined synapses turned the realm of unknown into unformation. Twisting invisible strings revert sharply perpetually turning feet, departing the ominous seeming structure and heralding you away from peril.

Flesch Reading Ease Score: 13.6

Very Easy to Read


Easy to Read


**Normal / Baseline **


Hard to Read


Very Hard to Read


4 Likes

This is really cool. Are you prompting the expected flesch reading score? The differences strike eerily similar to temperature settings.

Also, really like your application. Already looking good and can see lots of value in it. Are you planning on opening up your API?

1 Like

Thanks.

That didn’t work as well as I thought it would. Testing on ChatGPT Plus, it would not match score to the output. I think this is because it’s trained / prompted to be very easy to understand and won’t get technical without coaxing. So, I started giving it example text with a proper/real Flesch score… and it started outputting text … and a fake Flesch score saying it was perfect haha…

So, I broke down what each “level” of the Flesch score actually meant and translated… the prompt has grown to between 500 and 1000 tokens, though, which leaves less for the user to “attach” to the prompt, but it’s working well enough… hoping for a bigger context window eventually…

I need to nail down the output length better too. It’s usually around 300 words, but sometimes will do 50 or try to do 800 heh…

The differences strike eerily similar to temperature settings.

Temperature was something I forgot about changing too much as with ChatGPT it’s 0 - 2 not 0 - 1, and I usually set it around .8 and forget about it… Temperature is a lot more finicky in ChatGPT I’ve noticed… So, I’m still playing around to find the perfect balance… Temperature does affect how well it’s able to stick to the reading level, though…

Thanks! About a month of work (on and off here and there)… I’ve been thinking about an API for a while, but it’s probably not in the cards in the short term. Long term… maybe?

Appreciate the comment and kind words.

2 Likes

I went ahead and added Flesch reading ease score and other text stats …

Very cool. I believe you can actually get to a temperature of 2 using Davinci as well.

I’m really looking forward to the full release of your product! Wouldn’t be surprised to see its text in the game I play!

I don’t believe so? Or at least you couldn’t before… I could be wrong, though.

Final touches…

Nailing down the design / UI / UX… badly. Haha.

I ran one of the super-complicated texts through ElevenLabs… pretty impressive!

1 Like

@PaulBellow I don’t want to distract from your great work here, this looks awesome.

I’m new to language processing, don’t know much about the Flesch score. Is there a Python or NLP scoring tool that could be used to rate them automatically? If so, that would be really helpful in prompt development, I could see a path to automating prompt development.

The narrator is a bit stiff, but I’m very excited for the progress.
Loving the synergy! Good work!

Would love some more Dall-E images. There was so much beautiful spoken visual scenery in this video that Dall-E would love to gobble up

1 Like

IT’S ALIVE!

“In the eyes of this advanced AI writing generation system, you are never “just another client” or “just another project,” rather, you are somebody who values dynamic and authentic content. That is where I stride forward - it is who I am.”

Would you be willing to share the prompts that you are using to get it to adhere to the Flesch levels? The adherence you’re getting seems really nice. I haven’t been able to achieve that with mine so far…
Thanks!