Is there a real difference between system/user/assistant? We’ve been noticing that sometimes when we run a prompt we get back “undefined” (in nodejs). We’ve just been putting everything into the system role - should we be using user or assistant?
Traditionally, System is for ways for the model to act and primary instruction to follow, User is for the question/query and secondary instructions to follow and the Assistant role is for the AI’s replies to those questions and instructions.
The model has been tuned to respond differently to each and so usually you put them in their respective categories, although it’s perfectly valid to experiment with this and it may indeed produce novel and interesting, and possibly even useful results.
2 Likes
Gotcha - are there any quantifiable metrics around prompt performance if we use different roles?
It’s an interesting question, the answer to which will depend on your use-case. The entire science of AI at this level is still new and there are no dusty tomes of knowledge to refer back to, we are discovering it all as we go along.
If you made some roll performance metrics for your application they would be, if not the first, certainly among the first to be published and others would use them!