Response to query is not returning the expected json format and its content

Hi,

We are training the following type of dataset with GPT-3.5 Turbo. However, whenever we ask a query related to the dataset below, it modifies the assistance content and returns it. Sometimes, it simply returns plain information instead of the JSON data.

Let’s say I ask, “Please help me create a flow for triggering a command on a VM.” The expectation is that it should return the JSON data from the first entry provided in the dataset against the assistance content. However, it actually modifies the JSON format, does not return the nodes from that JSON, and also renames the name “START VM” to “Trigger VM”. The expectation is that it should return the same dataset as what we have trained it for. Could you please help me understand how to fix this issue?

{“messages”:[{“role”:“system”,“content”:“This flow executes a custom command as soon as the VM is started. After the Start VM step is complete, the command is executed, followed by sending an email notification to the requester.”},{“role”:“user”,“content”:“Could you assist me in creating a flow to execute a custom command when a VM starts?”},{“role”:“assistant”,“content”:“[{ ‘id’: 1, ‘name’: ‘Global Variable’, ‘children’: [2] },{ ‘id’: 2, ‘name’: ‘START VM’, ‘children’: [3] },{ ‘id’: 3, ‘name’: ‘Execute Command’, ‘children’: [3] },{ ‘id’: 4, ‘name’: ‘Email Notification’, ‘children’: }]”}]}

{“messages”:[{“role”:“system”,“content”:“This flow is designed to perform a Linux Patch, requiring necessary pre and post steps. It begins with stopping the application or APP, ASCS, DB, creating snapshots, and applying the Linux Patch. After stopping and starting the VM, DB, ASCS, and application, an email notification is sent to the requester.”},{“role”:“user”,“content”:“I need help creating a flow to perform a Linux Patch.”},{“role”:“assistant”,“content”:“[{ ‘id’: 1, ‘name’: ‘Global Variable’, ‘children’: [2] },{ ‘id’: 2, ‘name’: ‘STOP APP’, ‘children’: [3] },{ ‘id’: 3, ‘name’: ‘STOP ASCS’, ‘children’: [4] },{ ‘id’: 4, ‘name’: ‘STOP DB’, ‘children’: [5] },{ ‘id’: 5, ‘name’: ‘Create Snapshots’, ‘children’: [6] },{ ‘id’: 6, ‘name’: ‘Linux Patch’, ‘children’: [7] },{ ‘id’: 7, ‘name’: ‘STOP VM’, ‘children’: [8] },{ ‘id’: 8, ‘name’: ‘START VM’, '”}]}

To increase our own comprehension:

{
"messages": [
{
"role": "system",
"content": "This flow executes a custom command as soon as the VM is started. After the Start VM step is complete, the command is executed, followed by sending an email notification to the requester."
},
{
"role": "user",
"content": "Could you assist me in creating a flow to execute a custom command when a VM starts?"
},
{
"role": "assistant",
"content": "[{ 'id': 1, 'name': 'Global Variable', 'children': [2] },{ 'id': 2, 'name': 'START VM', 'children': [3] },{ 'id': 3, 'name': 'Execute Command', 'children': [3] },{ 'id': 4, 'name': 'Email Notification', 'children': [] }]"
}
]
}

{
"messages": [
{
"role": "system",
"content": "This flow is designed to perform a Linux Patch, requiring necessary pre and post steps. It begins with stopping the application or APP, ASCS, DB, creating snapshots, and applying the Linux Patch. After stopping and starting the VM, DB, ASCS, and application, an email notification is sent to the requester."
},
{
"role": "user",
"content": "I need help creating a flow to perform a Linux Patch."
},
{
"role": "assistant",
"content": "[{ 'id': 1, 'name': 'Global Variable', 'children': [2] },{ 'id': 2, 'name': 'STOP APP', 'children': [3] },{ 'id': 3, 'name': 'STOP ASCS', 'children': [4] },{ 'id': 4, 'name': 'STOP DB', 'children': [5] },{ 'id': 5, 'name': 'Create Snapshots', 'children': [6] },{ 'id': 6, 'name': 'Linux Patch', 'children': [7] },{ 'id': 7, 'name': 'STOP VM', 'children': [8] },{ 'id': 8, 'name': 'START VM', 'children': [] }]"
}
]
}

Assistant is what the AI said in past turns. You do not use it for data input.

Additionally, I don’t see anything in your instruction telling the AI what type of output it is supposed to be producing.

The system role should be a sort of “programming” the AI, telling the AI its identity, what role and task it is serving, and especially, a standalone section describing the type of output it should produce and how it should arrive there. Then an example of what it outputs given an input doesn’t hurt.

Then “create a flow for triggering” has very little meaning. If you just want the first element from a list, you really don’t need an AI to do that, but if you want it to perform like mylist[0], you need to tell it in uncertain terms the tedious menial task it shall perform.

Thank you @_j for inputs. are you suggesting to train a data like below ?

{
“messages”: [
{
“role”: “system”,
“content”: “As the AI assistant, your role is to generate a JSON representation of the requested flow based on the user query. Please ensure that the output adheres to the specified JSON format provided in the assistant’s response. For example, if the user needs help creating a flow to perform a Linux Patch, the output JSON format should resemble the following: [{ ‘id’: 1, ‘name’: ‘Global Variable’, ‘children’: [2] },{ ‘id’: 2, ‘name’: ‘STOP APP’, ‘children’: [3] },{ ‘id’: 3, ‘name’: ‘STOP ASCS’, ‘children’: [4] },{ ‘id’: 4, ‘name’: ‘STOP DB’, ‘children’: [5] },{ ‘id’: 5, ‘name’: ‘Create Snapshots’, ‘children’: [6] },{ ‘id’: 6, ‘name’: ‘Linux Patch’, ‘children’: [7] },{ ‘id’: 7, ‘name’: ‘STOP VM’, ‘children’: [8] },{ ‘id’: 8, ‘name’: ‘START VM’, ‘children’: }].”
},
{
“role”: “user”,
“content”: “I need help creating a flow to perform a Linux Patch.”
},
{
“role”: “assistant”,
“content”: “[{ ‘id’: 1, ‘name’: ‘Global Variable’, ‘children’: [2] },{ ‘id’: 2, ‘name’: ‘STOP APP’, ‘children’: [3] },{ ‘id’: 3, ‘name’: ‘STOP ASCS’, ‘children’: [4] },{ ‘id’: 4, ‘name’: ‘STOP DB’, ‘children’: [5] },{ ‘id’: 5, ‘name’: ‘Create Snapshots’, ‘children’: [6] },{ ‘id’: 6, ‘name’: ‘Linux Patch’, ‘children’: [7] },{ ‘id’: 7, ‘name’: ‘STOP VM’, ‘children’: [8] },{ ‘id’: 8, ‘name’: ‘START VM’, ‘children’: }]”
}
]
}
I tried to train the dataset suggested by you like putting the instruction but I received like in below screenshot. It gave me plain text response but I am expecting for json and should be from my trained dataset