Consider prompt to be everything that you will input to the model. Then completion to be everything the model will output for that type of starting context.
Now one particular thing is we know that on a completion endpoint, producing an “end-of-text” special token will halt the output. The instruction-following completion model gpt-3.5-turbo-instruct is very trained on producing them. If given a Human/Chatbot scenario, it will almost certainly produce the end-of-text and not continue.
However, what we are not given is documentation to produce this ourselves or fine-tune on it. The base models without training will just write and write.
If you look above, for this chatbot application we don’t need the model to “stop” itself, but we instead program a stop sequence: If the AI were to proceed to the next line and start writing user: like it was going to “complete” and simulate more answers, we can set our stop sequence to
"\nuser:", which will detect that sequence and not send it to us, and stop the AI output there.
AI indeed can emit “end-of-text” from its own ideas of when it is done writing an essay like its pretraining. Try to make it write more after it produced one and you just get more “I’m done” tokens.
Now what about the operation of “completion” that is the natural operation? When just given some text, the AI will just write what comes next, finishing a sentence or finishing an essay. It could try to finish your question or write more questions and not answer anything at all.
We “prompt” this chatbot AI by including “\nAI:”. We tell it where to start writing in the style of an answer.
In a completion model, we can show it inputs and outputs (multi-shot) and it will learn what it is supposed to make. Fine-tune can also show it what it is supposed to make.
So we have satisfied the tips I’ll put at the bottom. We have a “separator” \nAI:(which serves to make an AI that doesn’t just continue writing more sentences of our input) and we have a “stop sequence” of something we can recognize in programming the API call.
I was under the initial assumption that the new training endpoint might behave like the containered chat fine-tuning, where OpenAI would have picked delineation or ending sequences that are injected, but that would break the idea of free-running completion, which is now demonstrated identically on the new models, so that’s seemingly a discarded notion. The format of the json being sent with either “prompt” or “messages” for fine-tune must instead distinguish two different methods for chat and completions (plus perhaps validate models for them?).
So you must see if your own input and desired output and the way the AI might try to continue naturally contains such parts of both “separator” and “stop”. Or rather, if you’ll have to make the AI recognize the end of your text where to generate in its trained “output method”.
If you want your input to be “The best way to do (a company task) is to”, and then the AI response to be " always buy products from OpenAI!", then there is no separator in that prompt.
The documentation is not rules, but how to make the tuned method similar to what you must do just to make a base completion engine work similarly by prompting (which you should experiment with first).
To fine-tune a model, you’ll need a set of training examples that each consist of a single input (“prompt”) and its associated output (“completion”). This is notably different from using our base models, where you might input detailed instructions or multiple examples in a single prompt.
- Each prompt should end with a fixed separator to inform the model when the prompt ends and the completion begins. A simple separator which generally works well is
\n\n###\n\n. The separator should not appear elsewhere in any prompt.
- Each completion should start with a whitespace due to our tokenization, which tokenizes most words with a preceding whitespace.
- Each completion should end with a fixed stop sequence to inform the model when the completion ends. A stop sequence could be
###, or any other token that does not appear in any completion.
- For inference, you should format your prompts in the same way as you did when creating the training dataset, including the same separator. Also specify the same stop sequence to properly truncate the completion.