If it ain’t broke…right?
Just to clarify:
-
separator: The last part of your prompt. It tells the AI that your output format shall begin after a particular sequence is seen. It is the ultimate “prompting” to the AI.
-
stop sequence: The last part of your completion. It is a particular phrase and token sequence that when produced, will trigger the matching API’s “stop” strings, and terminate the likely continuing or repeating output.
Goofy completion model tricks:
Useful stop sequence fine-tuning
{“prompt”: “prefix:<prompt text>”, “completion”: “<ideal generated text>prefix:”}
You set the stop sequence as your same input role prompting. This is the pattern of chatbots, but most wouldn’t think to fine-tune this way.
Why: If you turn off the stop sequence, you get an AI that keeps on simulating these turns. You can see it make a bunch of fine-tuned responses to its own fine-tuned inputs.
Going past the prompt you trained on:
given normal fine-tune training:
{“prompt”: “<prompt text> #—>”, “completion”: “<ideal generated text>##STOP!##”}
You can disobey your own prompt style. This could have some interesting uses for probing performance.
Overcomplete prompt:
"A ripe banana#—>Sure, I’ll write a story
"I like frogs#—>{“sentiment”: “happy”, "length
You can go deeper into your fine-tune, and redirect the output to particular example, particular part of output, or even out-of-domain. See what is produced.
Incomplete Prompt:
“A ripe”
When the AI completes your own prompt, you see what the fine-tune is expecting from you, and how long before (and if) it makes the separator on its own by being strongly-tuned.