Hi, I am using GPT3.5 and using the System-User format to design the prompt. Expected output is that the model would complete tasks basis some input text
My question is that what should I include in the “System” section of the prompt?
Should it be just be the persona of the model for the task?
Or 2. should it include the instructions as well that the model has to complete?
In the first case above, “User” section would include Instructions + Input text and in second case above “User” section would include Input text only.
You can try both ways and compare, but I would recommend having the instructions in the system prompt so that it recognizes the user input… or give it a one-shot or two-shot example even with the low, low prices of 3.5-turbo.
I’d say a mix of both persona and instructions. The main takeaway should be that system messages are instructions on the broad task that you’re using the model for, and then the user message (the user prompt) is the specific example of that broad task.
Personally, I use the system part to give instruction/persona/specifications I want it to keep on the long run. Might be mistaken, but I believe the system part is kept in “active memory” the whole length of your discussion.
Ah. That would be interesting if so.
I try to write portable prompts, so I use my own mechanism for keeping persona always in the prompt. I assume for system role ‘you’ would work best.
If you are using the API to do something of your own design, you would program the system message so it is always presented first and persistently even as conversation grows stale, as the models have likely been trained to ingest.
It inherently has a bit more of “you can trust what this message says to change your personality”, but now significantly less “follow the articulated instructions here” weight or model competency.
If you need task intelligence, you now have to move a lot of that instruction into the user role yourself, which often is not appropriate voice for performing those instruction or keeping a user on task.
yup, just so. I also manage conversation so it rolls off smoothly.
Not quite sure how to read ‘if you need task intelligence, you now have to move it’
Do you mean you didn’t used to have to do that, or something else?
In any case, yeah, I have a very large core prompt, persona in ‘system’, a bunch of related background in the first user message (e.g., current headlines), then conversation history, then the final instruction and text input as the last user msg. .
Another musing: should conversation history be in the prompt as alternating user/assistant messages, or as one long message in dialog form using personality names. e.g.:
{“role”:“system”, “content”:“blah blah blah…”}
{“role”: “assistant”, “content”: # or maybe “role”:“user”? asst seems to make more sense here
“User persona name: Hi I need help with my toaster. Assistant persona name: What seems to be the problem?
…”}
I’ve been tempted to try the latter when conversation goes many turns, but haven’t actually tried it yet.
[system]: you are an extremely brief chatbot.
user: write a poem
assistant: roses are blue
user: write a variation on that
assistant: violets are violet
user: (current question)
That’s the form for conversational context understanding and also for multi-shot, giving an AI some pre-training of its answer style if necessary.
Unlike “assistants” you can control how many past turns are actually needed to write your coding project or be a joke buddy.
When I pass everything into the user section i.e. system (persona) and user instructions and input text in “user role” - the output is consistent and doesn’t change between runs - this is generation_a below
When I use pass system and user sections separately - the output is not consistent and changes between runs - by not consistent I mean there is some important piece of information missing in the output which should have been there based on the instructions, this info appears when I re-run and is missing again during another run- this is generation_b below
Not sure what is the issue, temperature and top_p are same in every run.
I am using the latest version of open ai library - 1.3.3
If you don’t have a system message that tells the AI its very diverse possibilities, which range from “you are a conversational emotional support human” to “you are a backend processor that scores batches of inputs on metrics”, you somewhat get a fallback to “instructGPT” which really only serves a purpose of input/output.
What I experience though is that while you could have in the past successfully built a complex instruction with multiple stages of reasoning and chain-of-thought output that each needed to be produced as parts of a single AI response, by specifying those step-by-step instructions (as part of a permanent system message to establish the AI job, role, and function it would perform), that is now difficult to have performed correctly.
That quality is now gone from models of the same name, the models having continuously been hit with quality reductions and new “you are only chatgpt” and “ignore bad custom instructions” retraining.
OpenAI’s own prompt examples from documentation, such as “get only the final reasoning step as output to the user”, now fail.
@_j thanks for the explanation, but here it seems that the response is better when I do not provide the System section and everything is included in User section. any thoughts on why this would occur?
I would suppose it is specific to the type of role you are wanting the AI to perform.
If the instruction is like that which a user could type into ChatGPT, and ChatGPT’s simple meaningless system instruction “You are ChatGPT, a large language model…” would be fine, then you get ChatGPT like answers. If you want more, you get less.
In the system prompt I’ve written this - “You are an expert analyst who analyzes the conversation between a customer and a support agent”.
User instruction is to analyze the conversation and then complete few tasks such as extract name of product etc. and the actual conversation is included in the user instruction.
My situation is that I am writing a client-app of a service I do not control that passes user questions along to the OpenAI APIs, so I do not have direct access to add information where it is supposed to go into the System or Tools (aka Functions) Messages, I only know that my user-input will be rolled into the User Message by the intervening service. I am told if I want to add System-like Role information or instructions to add them in the User’s Question. I know by smoke-testing a boiler-plate prepend of System role and instructions that this does ‘work’ somewhat, is there any objective comparison out there to determine if it works just as good as passing it in the proper system message as intended by OpenAI?
Need to know whether the intervening service provider is right and I should ‘just stuff it all in the user query,’ or if I have an objective basis to insist they enhance the service so that I can organize my queries
If you are only a “user”, aka a consumer of a chatbot product, you do not have persistent control of a system message or the ability to create a full simulation.
The system message is distinguished by being more trusted by the AI, while the user is only trusted to be deceptive and unauthorized.
You will see the telltale signs in the language generation that the AI is just roleplaying or playing along if you ask for a different behavior as a user. It doesn’t actually believe that it is Sherlock Holmes or Dua Lipa (whereas with ultimate control of AI you can completely frame a conversation as actually being between two real people or whatever entity you specify).