Fine Tuning OpenAI Davinci - Stop Sequences not working

API
1

I am finetuning OpenAI Davinci to create synthetic data. The idea is to have a Chatbot where I ask questions and the model answers my question. But I have the problem that my model does not exclusively answer my question; it answers the question and then asks questions itself, which it answers in turn.

The JSON-File I have used looks like this:

'[{"prompt": "Ich muss mich nochmal entschuldigen, ich habe deinen Vornamen vergessen.<FRAGE>", "completion": "[Vorname].<ANTWORT>"}, {"prompt": "Bitte?<FRAGE>", "completion": "[Vorname].<ANTWORT>"}, {"prompt": "[Vorname]?<FRAGE>", "completion": "Ja.<ANTWORT>"}, ...]

So every “prompt” has a stop sequence at the end, called “”. Every completion sequence has a “” stop sequence.

I have started fine-tuning the model with my data like this:

## Start fine-tuning
subprocess.run('openai api fine_tunes.create --training_file prepared_data_prepared.jsonl --model davinci --suffix "MyModel_1205"'.split())

After fine-tuning is done, I want to try out the model with the following Code:

import os
import openai


pred_prompt = "Wie geht es dir?"
openai.api_key = os.getenv("OPENAI_API_KEY")

response = openai.Completion.create(
  model="MY_MODEL_PATH_HERE",
  prompt=pred_prompt,
  temperature=0.7,
  max_tokens=256,
  top_p=1,
  frequency_penalty=0,
  presence_penalty=0,
  stop=["<ANTWORT>"]
)

Generally, the script works, but it does not only answer the first question; the model continues to ask and answer questions, and that’s not what I like:

print(response["choices"][0]["text"])
'\n\nGut.\n\n(unverständliches Zeug)\n\nJa.\n\nHast du heute schon etwas gegessen?\n\nNein.\n\nSonst nicht?\n\nNein.\n\nSchläfst du gut in der letzten Zeit?\n\nJa.\n\nHast du den Schlaf gut heruntergeholt?\n\nJa.\n\nAuch wenn du müde warst?\n\nJa.\n\nAuf einer Skala von eins bis zehn, wie müde fühlst du dich denn in den letzten zwei Wochen insgesamt?\n\nDrei.\n\nAn welchen Tagen bist du am müdesten?\n\nAm Montag und am Dienstag.\n\nAn welchen Tagen bist du am unwohlsten?\n\nDas ist ganz unterschiedlich. Manchmal bin ich am Montag und am Dienstag am unwohlsten. Dann wieder am Mittwoch und am Donnerstag.\n\n'

So it seems like something is not working with my stop sequences, but I am not sure what I am doing wrong here.

Does anyone have suggestions?
Also it would be nice to create a Chat Interface in the Notebook, any suggestions here?

Thanks in advance!

2

there is also fullstop(.)
try this

presence_penalty=0,

stop=[“.”]

1 Like