Hi,
I don’t know, if this is a bug.
I fine-tuned a model and evaluated with 5-fold cross-validation. In one evaluation set, a quite large sentence is part of the input. If I use the system prompt for prompting this fine-tuned model with the evaluation set, the model output results in an invalid JSON entry for that sentence with around 3000 entries and you can see that the latest “normal” entry is repeated again and again and again. Is the sentence too large for the prompting? It consists of 142 tokens and 794 characters. Is there a limit that is reached? I repeated this test three times and the error occures every time. If I do the same without the system prompt, the error doesn’t show up.
Hi there and welcome to the Forum!
In order for us to take a closer look, it would be useful it if you could share more information about your fine-tuning project including purpose of the fine-tuning and ideally an example of your system, user and assistant messages.
Hi,
thank you for your reply. The purpose of the fine-tuning is the extraction of specified element types and phrases from a requirement.
Here is an example message:
{"messages": [{"role": "system", "content": "Classify the requirement whether it contains the element types activity, component, data, entity, state or not. Extract the indicative phrases for the element types in the requirement. An activity is defined as an action that is executed by a specific entity that can process data and communicate with other activities. A component is defined as a part of a software that provides functionality for a certain purpose and comprises one or more entities. A component can be deployed on a node. Data is defined as a collection of values that convey information. An entity is defined as a physical actor, software object, or external system that can hold data. A state is defined as the state of an activity or entity that is represented by data. List all the elements of all element types in a JSON array, providing for each entry the keys indicativePhrase and elementType. If there is no element type in the requirement, return an empty list."}, {"role": "user", "content": "The system should only have the bare minimum of information (Token validated) about the user of the charging station. "}, {"role": "assistant", "content": "[{\"indicativePhrase\":\"information\",\"elementType\":\"data\"},{\"indicativePhrase\":\"token\",\"elementType\":\"data\"},{\"indicativePhrase\":\"user\",\"elementType\":\"entity\"},{\"indicativePhrase\":\"validated\",\"elementType\":\"state\"}]"}]}
If I now send a request with the whole content of the system prompt and the requirement in the user prompt, it first extracts valid elements out of the requirement and then just keeps repeating the entries.
Let’s format.
{
"messages": [
{
"role": "system",
"content": "Classify the requirement whether it contains the element types activity, component, data, entity, state or not. Extract the indicative phrases for the element types in the requirement. An activity is defined as an action that is executed by a specific entity that can process data and communicate with other activities. A component is defined as a part of a software that provides functionality for a certain purpose and comprises one or more entities. A component can be deployed on a node. Data is defined as a collection of values that convey information. An entity is defined as a physical actor, software object, or external system that can hold data. A state is defined as the state of an activity or entity that is represented by data. List all the elements of all element types in a JSON array, providing for each entry the keys indicativePhrase and elementType. If there is no element type in the requirement, return an empty list.",
},
{
"role": "user",
"content": "The system should only have the bare minimum of information (Token validated) about the user of the charging station. ",
},
{
"role": "assistant",
"content": '[{"indicativePhrase":"information","elementType":"data"},{"indicativePhrase":"token","elementType":"data"},{"indicativePhrase":"user","elementType":"entity"},{"indicativePhrase":"validated","elementType":"state"}]',
},
]
}
Assistant message:
[
{"indicativePhrase": "information", "elementType": "data"},
{"indicativePhrase": "token", "elementType": "data"},
{"indicativePhrase": "user", "elementType": "entity"},
{"indicativePhrase": "validated", "elementType": "state"},
]
Recommendation if prompting the behavior:
Kidnap me and start yelling instructions at me and you will find lower performance than if I know my job and position in the hierarchy of the slave trade.
The AI is the same.
AI identity and purpose
You are xxx, employed by xxx
You are placed in an operational workflow as a xxx
You are operating on input data from xxx,
You are providing output data to xxx,
Clarify the system message so that the purpose of the assistant becomes extremely straightforward to human, and the poor 3.5 may follow. A massive token dump if the AI just is not trained well.
Task Overview
- Objective: The primary goal is to classify and extract specific information from requirements based on defined element types. The element types to be identified are activity, component, data, entity, and state.
Element Type Definitions
- Activity:
- Definition: An action executed by a specific entity that can process data and communicate with other activities.
- Component:
- Definition: A part of software that provides functionality for a specific purpose and includes one or more entities. It can be deployed on a node.
- Data:
- Definition: A collection of values that convey information.
- Entity:
- Definition: A physical actor, software object, or external system capable of holding data.
- State:
- Definition: The condition of an activity or entity as represented by data.
Instructions for Classification and Extraction
Identify Element Types:
- Analyze the requirement to determine if it contains references to any of the defined element types (activity, component, data, entity, state).
Extract Indicative Phrases:
- For each identified element type, extract phrases or sections of text from the requirement that indicate the presence of that element.
JSON Array Formation:
- Create a JSON array to list all identified elements.
- For each element, provide two keys:
indicativePhrase: The text snippet or phrase that indicates the element.elementType: The type of element identified (activity, component, data, entity, state).Handling Non-Identifiable Elements:
- If the requirement does not contain any identifiable element types, return an empty list.
Important: You must produce JSON objects with
"indicativePhrase": [activity | component | data | entity | state] that preserve the original document order of the phrase.
- Starting at the beginning of user input, you will make only one pass through the document, and only outputting
"indicativePhrase"again if it matches a criteria and appears after the previously-produced"indicativePhrase".- The JSON container is closed and response is ended when document is thusly processed.
This structure ensures clarity in understanding which elements are present in the requirement and the basis for their classification. The last gives a method to end the JSON so it doesn’t ramble on.
Training without much system instruction would have been a good employment of fine-tuning to save tokens. Here, you double-down on the following, but have a behavior only activated by the path through that same system prompt with sequences trained over and over.
Fine tuning may in fact increase the likelihood of new “indicativePhrase” so high that the AI simply can’t stop. Frequency penalty can make more of them less likely. logit_bias can tune the probability of closing a JSON.
1 Like
First of all thank you for your detailed reply!!! I really appreciate your help and explanations!
Okay, so as far as I understand, you don’t think that using the system prompt increases the likelihood of a strange behaviour of the fine-tuned model? I understand that your advise is rather to use a different prompt (regarding the content). Am I right?
Or to use the frequency_penalty or logit_bias?