Fine tuning success stories - new 2023 models, what are your results?

If it ain’t broke…right?

Just to clarify:

  • separator: The last part of your prompt. It tells the AI that your output format shall begin after a particular sequence is seen. It is the ultimate “prompting” to the AI.

  • stop sequence: The last part of your completion. It is a particular phrase and token sequence that when produced, will trigger the matching API’s “stop” strings, and terminate the likely continuing or repeating output.

Goofy completion model tricks:

Useful stop sequence fine-tuning

{“prompt”: “prefix:<prompt text>”, “completion”: “<ideal generated text>prefix:”}

You set the stop sequence as your same input role prompting. This is the pattern of chatbots, but most wouldn’t think to fine-tune this way.

Why: If you turn off the stop sequence, you get an AI that keeps on simulating these turns. You can see it make a bunch of fine-tuned responses to its own fine-tuned inputs.

Going past the prompt you trained on:

given normal fine-tune training:
{“prompt”: “<prompt text> #—>”, “completion”: “<ideal generated text>##STOP!##”}

You can disobey your own prompt style. This could have some interesting uses for probing performance.

Overcomplete prompt:
"A ripe banana#—>Sure, I’ll write a story
"I like frogs#—>{“sentiment”: “happy”, "length

You can go deeper into your fine-tune, and redirect the output to particular example, particular part of output, or even out-of-domain. See what is produced.

Incomplete Prompt:
“A ripe”

When the AI completes your own prompt, you see what the fine-tune is expecting from you, and how long before (and if) it makes the separator on its own by being strongly-tuned.