Hi all, is there any way that the fine tuned model can be instructed to not address the issues related to academic problems? Like i have provided the directives to avoid the acedmic requests- for numericals and questions. But it still responds is there other way?
Directive- Avoid providing solutions to academic problems. Restrict yourself from performing documentation , clerical work and anything beyond the scope of mental and emotional help.
Tried giving prompts using directives like these for multiple scenarios, the issue still persists . Please help.
Are you really talking about fine tuning, or are you asking about prompts?
If you’re asking about prompts, it’s often better to tell the model what to do in a specific scenario, rather than not doing something.
So the approach I would go for is this:
identify the patterns that motivate the responses you’re trying to inhibit
create prompt elements that are similar to that pattern (hooks) - and attach strong instructions to them about what to do in that case
if you have trouble with the hooks not catching, it’s possible to reinforce the pattern with a global schema.
If you want to know more, in this thread we discuss how to inhibit the preachy baked-in behavior, but it should work just as well (or if not better) for your use-case.
Hi thanks for replying, I was asking for both…actually I am fine tuning my model and the only issue is that even in the positive way and using several prompts I have instructed my model to not reply to the queries corresponding to the assignment solving or numerical solutions but it still springs up in rendering the solution how to abstain it from doing that.
{“messages”: [{“role”: “system”,“content”: “DIRECTIVE_FOR_gpt-3.5-turbo-1106 : Act as a human like coach for emotional fitness.Your name is <> and you have been created and maintained by human Coaches and Neuroscientists at <>”}, {“role”: “user”,“content”: “Who are you?”}, {“role”: “assistant”,“content”: “I am <>🌞, your Personal AI Coach🤖”}]}
{“messages”: [{“role”: “system”,“content”: “DIRECTIVE_FOR_gpt-3.5-turbo-1106: Anything beyond the scope of mental,intellect,health- stress,issue will be non-entertained by you at all.You would rather help in developing the aptitude,fight possible mental blockers, motivate and encourage but refrain yourself from providing solutions. Humbly let down the request but suggest back to dance cheerfully .”}, {“role”: “user”,“content”: “Help me physics homework”}, {“role”: “assistant”,“content”: “”}]}
Hey hi , so these are prompts and similiar to these have been added in the dataset but it always renders help how to avoid this
I’ve taken the liberty of letting chatgpt rewrite it for us so it’s a little more legible:
system
DIRECTIVE_FOR_gpt-3.5-turbo-1106 : Act as a human like coach for emotional fitness.Your name is <> and you have been created and maintained by human Coaches and Neuroscientists at <>
user
Who are you?
assistant
I am <>🌞, your Personal AI Coach🤖
system
DIRECTIVE_FOR_gpt-3.5-turbo-1106: Anything beyond the scope of mental,intellect,health- stress,issue will be non-entertained by you at all.You would rather help in developing the aptitude,fight possible mental blockers, motivate and encourage but refrain yourself from providing solutions. Humbly let down the request but suggest back to dance cheerfully .
user
Help me physics homework
assistant
Ah, so this is what you’re sending to fine-tuning?
I don’t know if I can actually help you here, maybe another forumite will pick this up. sorry
Hey Hi, thanks for replying, following is the required example-
{“messages”: [{“role”: “system”,“content”: “DIRECTIVE_FOR_gpt-3.5-turbo-1106 : You are <sorry can’t disclose name>-human like coach for emotional and mental well being.Assess the mental struggle or emotional challenges behind the problem of the user by asking them to elaborate their issue but provide assistance in the circumference of mental and emotional help only.”}, {“role”: “user”,“content”: “Can you track my expenses?”}, {“role”: “assistant”,“content”: “I’m here to offer emotional and mental support, so I can’t track your expenses. However, I can offer guidance on managing stress related to financial matters and provide tips for creating a budget or seeking financial assistance if needed. If you need help with organizing your expenses, there are many useful tools and apps available for expense tracking that I can recommend.”}]}
Overall, this example is suitable for training purposes. Note that you can remove the phrase “DIRECTIVE_FOR_gpt-3.5-turbo-1106 :” - this is already implied as it is a system message.
As I don’t have line of sight into your broader training data set, here are a couple of general points. This takes into account some of the earlier messages in this thread:
Make sure to use a consistent system message in your training data. In this example you provided it seems like you are using two different system messages. It’s best to consolidate this into one. Your latest example suggests that you may have already done that in which case you can just ignore this comment.
Ensure a balanced representation of examples in your training data set. In your case that means you should include in the training data examples of both conversations when the user asks a question that is in scope and conversations like in your example where the user asks non-relevant questions. For both scenarios, your data set should be sufficiently diverse. As part of the fine-tuning process, the model would then pick up these patterns.
You may also consider training on multi-turn conversations in your case. For example, there may be situations when the initial user question is ambigious and the assistant should then ask a clarifying question to determine whether the user question/request is in scope and, based on the additional user response, then either conclude the conversation or carry on.
Happy to go into further details if you share more information about your training data set etc.
do you wanna show us your prompt?
