Hi, I fine-tuned the gpt-3.5 on around 400 tweet examples. Upon trying to the trained model in the playground it gives empty assistant responses. Did I overlook something?
1 Like
It is possible that you overlooked something.
Answering what that is without more information is more difficult.
Can you easily switch between gpt-3.5-turbo and see it working, and then your model and see that it doesn’t produce a response? You have the maximum output slider higher than 1?
Do you have a success status for your model in the fine-tune user interface next to the playground button in the API platform top bar?
Do you have a system prompt in training examples that must be emulated, and then you also are trying to activate on a similar user input to receive a response?
If you interface with the model with a script, you can see the actual bytes being sent by the model, and the finish_reason to understand why there was a termination.
1 Like
Hi and welcome to the Developer Forum!
Can you give an example of your training set (a few lines is fine) and also a code snippet of your API calling code and any setup it may rely upon.
1 Like
If you interface with the model with a script, you can see the actual bytes being sent by the model, and the finish_reason to understand why there was a termination.
Yes, this is the response I am getting
<OpenAIObject chat.completion id=chatcmpl-85v6CbSkiaKgcqayUWNZ7xXHERbyR at 0x130cfab10> JSON: {
"id": "chatcmpl-85v6CbSkiaKgcqayUWNZ7xXHERbyR",
"object": "chat.completion",
"created": 1696421820,
"model": "ft:gpt-3.5-turbo-0613:sintra::85tnrldY",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": ""
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 23,
"completion_tokens": 0,
"total_tokens": 23
}
}
Can you easily switch between gpt-3.5-turbo and see it working, and then your model and see that it doesn’t produce a response? You have the maximum output slider higher than 1?
Yes
Do you have a success status for your model in the fine-tune user interface next to the playground button in the API platform top bar?
Yes
Do you have a system prompt in training examples that must be emulated, and then you also are trying to activate on a similar user input to receive a response?
These are a few examples of the training data. Now I see the issue might by that there is no user role input.
{"messages": [{"role": "system", "content": "write a tweet"}, {"role": "assistant", "content": "Most movies I\'ve seen, I\'ve seen only partially, in passing, by e.g. sticking my head in the door for a few minutes as someone was watching it, or seeing it out of the corner of my eye on someone else\'s screen on a plane."}]}\n
{"messages": [{"role": "system", "content": "write a tweet"}, {"role": "assistant", "content": "Note the date."}]}\n
1 Like
Hi,
I am just prompting it on the playground now. This is the data.
{"messages": [{"role": "system", "content": "write a tweet"}, {"role": "assistant", "content": "Most movies I\'ve seen, I\'ve seen only partially, in passing, by e.g. sticking my head in the door for a few minutes as someone was watching it, or seeing it out of the corner of my eye on someone else\'s screen on a plane."}]}\n
{"messages": [{"role": "system", "content": "write a tweet"}, {"role": "assistant", "content": "Note the date."}]}\n
1 Like
You might have made a model than only writes random tweets from a system message if it does work…
1 Like
Even then it doesn’t seem to work:
response = openai.ChatCompletion.create(
model="<id>", messages=[{"role": "system", "content": "Write a tweet about a teddy bear"}], temperature=1
)
print(response["choices"][0]["message"]["content"])
Gives empty response.
1 Like
And you have no stop sequence parameter specified?
You can try a system prompt “You are ChatGPT, a large language model” and see if there is any brain there at all.
Unfortunately you can’t demote the “end of role” token that might be emitted by the AI and could immediately end the output. An empty “user” and it might have been trained on producing the user container anyway, triggering the chat internal “stop”.
1 Like
Seems to do something.
Unfortunately you can’t demote the “end of role” token that might be emitted by the AI and could immediately end the output. An empty assistant and it might have been trained on producing the assistant container anyway, triggering the chat internal “stop”.
Mind elaborating?
1 Like
The ChatML container that messages are placed within uses special token numbers that the user can’t reproduce that delineates the roles.
<|im_start|>system<|im_sep|>You tweet nonsense<|im_end|><|im_start|>user<|im_sep|><|im_end|>
<|im_start|>assistant<|im_sep|>(AI writes here and makes im_end)
The im_end is one of the stop tokens (100260, 100265) that will terminate the output. The training method, being undocumented, may have unexpected outcomes like you see without the example roles being filled.
In the playground, you show an empty user message and you wrote as the assistant. Same thing there about using roles correctly, but at least you see it is not broken unless you give it your identity.
You can send a “delete model”. And then rewrite your examples so that you show the type of user input that should evoke a particular type of output.
(and ultimately, “write a tweet” is not a task that requires a fine-tune model that costs eight times as much)
1 Like
In the playground, you show an empty user message and you wrote as the assistant. Same thing there about using roles correctly, but at least you see it is not broken unless you give it your identity.
Funnily enough, the first assistant message was generated by the model. I wanted to replicate having system message, but no user message.
(and ultimately, “write a tweet” is not a task that requires a fine-tune model that costs eight times as much)
Sure, just playing around.
So the conclusion is to retrain, but include user message?
1 Like
Yes, you’ll need to come up with the inputs that would make the desired outputs.
Here is an example of one training conversation, that you would then enclose in the single line training format. It has:
- The unique system identity for your fine-tune use
- A typical input you would expect from a user
- The new fine-tune response you expect from your new model
System: You are ROBO
User: Write tweet: OpenAI introduces new image generator dall-e
Assistant: Jeez, will you look at this crazy shizz. Make a computer picture, who needs that? OpenAI has made being lazy easy. Learn to draw fools! dalle
Because you put the new assistant behavior into the model’s new training, triggered by just its new identity, you no longer must include personality rules in a long prompt to make it act that way.
2 Likes