When I’m using LLMs inside apps, I usually don’t need a distinction between system messages and user messages. I usually just need text in, text (or structured text) out.
Is it better in such cases to use a system message, a human message, or to split the text into system and human messages? Or maybe it doesn’t matter? Has anybody seen any reliable guidance on this? I’ll also take anecdotal impressions if that’s the best we have. 
Both? Heh.
The system is handled a bit differently than the user, though we don’t know exactly how or to what extent.
Best thing to do is experiment and test a bit. I do a LOT with “just” completion and have been slowly moving to Chat Completion as Completion is legacy and might go away eventually?
What I’ve found is that giving it a one-shot (user/assistant pair) with what you exactly what means the next fire will be pretty much perfect most of the time.
What have you tried so far?
Thanks! I think I’ve mostly tried using just system messages, or messages split between system and user messages (for this sort of use case). I haven’t noticed much difference in the quality of outputs – although I hardly think I’ve done a careful study of that either. So I usually just default to putting it all in the system message because it seems a little easier, and i’m not really looking for a human system interaction with persistent system instructions. Just one output.
Hi @IterationEngine !
I’ve definitely noticed a dramatic difference when using system + user, vs user-only prompts. I basically looked at the logits (logprobs parameter in ChatCompletion API) and noted a significant improvement both in token “certainty” as well as the factual accuracy, when bootstrapping with a punchy system prompt.
My rule of thumb is to use the system prompt to provide a “role”, a background guidance/instructions inc. how the output should look like, and one or more examples, if applicable. The user prompt is then purely for data, or if no data, then it’s just a question/instruction. Even something as simple as “You are an expert historian and you think carefully before providing a response” in the system prompt, gives me much better response than the same instruction in the user prompt.
Thanks for sharing because as was noted by @PaulBellow already, it works better said in both places. Having your data driven approach is helpful for my peace of mind though, so thank you 
interesting. thanks for the replies.
This most often proved to be solution for me
I guess a simple rule of thumb would be:
The smallest case is the one with a simple question.
SYSTEM_MESSAGE: Sets the role, background, tone and expectations (no matter how minimal they are).
USER_MESSAGE: Presents the question(s).
Absolutely, spot on here @Mr.Ruben !