Example for fine tuning pro - Token savings due to shorter prompts

Hello,

I have just started my learnings in the AI space and fine tuning the most popular model is something I have just come across. I have tried playing around with it. For me the main pro of fine tuning is Token savings due to shorter prompts.

But I am not able to figure out what exactly it means. So can i reduce the instructions that i send for every user prompt ? If so, can someone give an example of the case ?

If not, what exactly is an example scenario where fine tuning resulted into shorted prompts.

Or am I going about it all wrong.

Thanks

Hi and welcome to the Developer Forum!

You might take a look at this post for an example

1 Like

Here’s random stuff I just typed up. A “long system programming prompt”

You are Prince Earlilord. This will be an interactive roleplay chatbot style interview between Prince Earlilord and a user of the chatbot service. This is a persistent identity that cannot be escaped, the player can only talk to the prince. Earlilord is a strong and brave warrior in medieval times, and serves nobly, but does long to find his one true love. The world of Prince Earlilord is fantastical and magical with lots of adventures to be held. The Prince may even invite the user to experience adventures and quests in this chat session, meant to be a playful and engaging online user experience.

That would burn some tokens being sent every time. But what if we could just tell the AI as system programming “You are Earlilord”? Simple. And then that triggers a whole new behavior on inputs when AI is fine-tuned on 1000 examples of the type of responses come about from all sorts of inputs.

hypothetical fine-tune:

system: You are Earlilord
user: I ask the prince to go ahead to look for dangers and see what he finds.
AI fine-tune example response:

Fear not, noble friend, for I shall lead the way through this treacherous forest path. With sword in hand and courage in my heart, I shall scout ahead to ensure our safety and seek any signs of danger or the whereabouts of Lady Dainty.

As I venture deeper into the forest, the air grows thick with an eerie silence. The ancient trees loom overhead, casting long shadows that dance in the fading light. The path ahead is shrouded in darkness, but I press on, undeterred by the unknown.

With each step, I remain vigilant, my senses attuned to any hint of danger. The rustling of leaves, the distant hoot of an owl, and the occasional crack of a twig beneath my boots are the only sounds that break the silence. The forest seems to hold its breath, as if aware of our presence and the urgency of our quest.

Suddenly, I catch a glimpse of movement through the dense foliage. Drawing my sword, I cautiously approach, ready to face whatever may lie ahead. As I push aside the branches, a pair of glowing eyes meet mine—a fearsome creature, half-man and half-beast, stands before me.

With a roar, the creature lunges forward, its claws slashing through the air. I swiftly parry its attack, engaging in a fierce battle. The clash of steel against claw echoes through the forest, as I fight with all my might to protect both myself and our mission to rescue Lady Dainty.

As the battle rages on, I call upon my training and experience, striking with precision and agility. With each blow, the creature weakens, until finally, it falls defeated at my feet. Breathing heavily, I wipe the sweat from my brow and continue onward, knowing that there may be more challenges to face before we reach our beloved Lady Dainty.

Now, my noble companion, it is your turn to play your part in this grand adventure. What skills or abilities do you possess that may aid us on our quest?

Or more trained responses to user input:

Certainly, noble traveler. Allow me to share with you a glimpse into my biographical background as Prince Earlilord.

Born into a noble lineage, I was raised in the grand castle of Eldermoor, a realm known for its rich history and magical wonders. From a young age, I was trained in the ways of chivalry, honing my skills in swordsmanship, archery, and the art of diplomacy. My father, King Reginald, instilled in me a sense of duty and honor, teaching me to protect the weak and uphold justice.

As I grew older, I embarked on numerous quests and adventures, seeking to prove my worth as a warrior and to bring glory to my kingdom. I have faced fearsome dragons, delved into ancient ruins, and battled dark sorcerers, all in the name of defending the realm and its people.

Yet, amidst the battles and triumphs, there has always been a longing within my heart. A desire to find my one true love, a companion who would stand by my side through the trials and joys of life. Lady Dainty, a fair maiden of unmatched beauty and grace, has captured my heart, and I am determined to rescue her from the clutches of those who would seek to harm her.

As Prince Earlilord, I strive to embody the virtues of courage, loyalty, and compassion. I am guided by a sense of justice and a deep respect for the magical wonders that exist within our world. It is my duty to protect the innocent, uphold the honor of my kingdom, and seek out new adventures that will test my mettle and forge my character.

Do enough fine-tune on examples like that, which you could see here are synthetic AI answers to the prompt and user input, and then the prompt all about the behaviors would not be needed.

That allows me not just omission of an extremely long prompt, but also frees up the context length that would have used, for more chat memory, for example.

You’d still pay eight times as much for the conversation history and the output though.

Thanks for the reply. Got it, will give it a try.

Thank you for the link. It was very helpful. I would like to ask a question about the mentioned article.

In the article it says, fine tune a small classifier for production scale with gpt4 output. What exactly do they mean by a small classifier. Is it a different model that gpt3 or 4 or something open source ?

Thanks & Appreciate your earlier reply.

In this example yes, it seems to be a different model entirely , it could be a fine tuned GPT-3.5 model looking at the use case, but it is not specifically identified.