Hi everyone,
I’m trying to build a chatbot that is supposed to simulate different patients (I have the data as questions of a doctor and answers of a patient) for medical students to practice asking questions.
I have used a small subset of the data (a conversation with just 1 patient, ~40 prompt-completion pairs) to fine-tune text-davinci-003 to answer the questions according to this conversation. The model gives good answers in the first line, but then starts producing extra lines (both from the “doctor”/user side and the “patient” side) and I am not sure why it is doing so (as if it is trying to continue the “text” of the conversation and use all of the tokens). Most often it’s the next line from the “doctor” side, but sometimes it can give 2-3 lines from the patient side (they all start from “AI:”).
When I replace the fine-tuned model in my code with a standard text-davinci-003 it works absolutely fine (gives only 1 line/response as intended and then stops and waits for the next input from the user). I cannot put another screenshot here, so you’ll have to trust me.
Any suggestions as to why this is happening and how it could be fixed I would greatly appreciate!
Can you share some of your training data?
You can also use stop sequences to prevent it from happening
https://help.openai.com/en/articles/5072263-how-do-i-use-stop-sequences
3 Likes
Hi,
Thanks for your reply.
My dataset looks like this:
Here’s a link to it on my gdrive.
I have many more scenarios like this (with more or less the same prompts, but different completions) and I want the chatbot to recreate these scenarios.
I have ‘Human:’, ’ AI:’ in my stop sequences, but it doesn’t help. I tried adding ‘\n’ as well, but it gives me an error, will try to figure out why.
Code:
try:
response: dict = openai.Completion.create(
model=‘davinci:ft-personal-2023-03-12-15-39-51’,
prompt=prompt,
temperature=0.3,
max_tokens=25,
top_p=1,
n=1,
frequency_penalty=0,
presence_penalty=0.6,
stop=[’ Human:‘, ’ AI:’]
)
In your training files you should be using a separator between the prompt and completion
I believe using “\n” as a stop is broken right now, based on a recent OpenAI staff post.
{“prompt”:“Subject: Update my address\nFrom:Joe Doe\nTo:[my email]\nDate:2021-06-03\nContent:Hi,\nI would like to update my billing address to match my delivery address.\n\nPlease let me know once done.\n\nThanks,\nJoe\n\n###\n\n”, “completion”:" 4"}
Where \n\n###\n\n is the unique identifier to indicate a separation.
It’s quite possible that without this separator it’s attempting to continue the conversation.
It’s strange that the stop sequences aren’t working. They are a bit finnicky; it seems that because of ChatML, they are being worked on.
If you haven’t gone too far in your training I’d recommend adding a separator in the middle.
Oh. I’ve just noticed. You have whitespace in front of your stop sequences. The needs to be removed and are definitely the culprit
["AI:", "Human:"]
After removing the whitespace, the stop sequences should work
1 Like
Thanks, removing the whitespaces from the stop sequences worked indeed.
It started giving two white lines after replying.
I’ll try adding a separator between the prompt and the completion and see if it has to do anything with it.
Thanks a lot for your help!
Davinci usually will add newlines to indicate a finished thought. Having “\n” as a stop was pretty much essential to using it before ChatML was released.
If you are going to re-do your training files. Perhaps using [END] instead of “\n” may be a better solution as well? Hard to say based on the fact that everything changes so much and there’s no sort of communication before it happens. All I know now is that using “\n” is broken.
Also, last I used fine-tuning it actually scolded me for using such a token-heavy separator like “\n\n###\n\n” even though it’s in the documentation.
Another option is “->”. Although with a small amount of training data it may corrupt it (because GPT recognizes the token as something else).
Taken from the Fine-tuning docs
With a sufficient number of examples, the [type of] separator doesn’t make much of a difference (usually less than 0.4%) as long as it doesn’t appear within the prompt or the completion.
1 Like
