The new 3.5 model (0613) is supposed to be better at following the system prompt

Prompting
,
1

What is y’alls experience with the system prompt as an alternative to including more instructions as part of the user prompt? I’ve done a little bit of work with it, but do not see significant improvement.

2

With 0613 models it works really well, most of the changes people are finding issue with is down to this, if they move their prompt setup text into the system prompt it will perform very similar to 0301

1 Like
3

Have you had any experience with the number of instructions 0613 can follow from the system prompt? My intuition is that less is more, and if I can get the job done with fewer instructions, that’s better, but I’m curious to see what others have experienced with regards to large system prompts.

4

I never give more than 4 instructions in any prompt, and if I can, just 1

1 Like
5

I have had great success with the system prompt. In the system prompt, I invoke a persona related to the task and describe the quality of work the persona produces. I also add rules and checks for the user message. The user message is more conversational and describes the specific task.

I observed:

  • Better adherence to rules in the system prompt
  • More creative responses - able to generate good, valid alternatives to the presented answer
  • Refusing to answer user prompt - following a rule to check conditions before answering

I am still exploring the temp/top-p settings. temp=1 and top-p=1 usually gives me good results, but if a particular task is resulting in repetitive answers, I lower the temp by 0.2-0.3 and that usually works.

1 Like
6

I try not to ramble in the system message - longer instructions do seem to be ignored/forgotten somewhat ( also the token limit! )
But my user prompts can be quite large, using Better Chain of Thought - but the results remain exceptional.

The main difference between using the bCoT in the Completions API and Chat API is better adherence to rules in the Chat API as mentioned in my other reply. Usually I have:

  • The details of the bCoT in the system prompt and the steps in the user message
    OR
  • High level steps in the system prompt and detailed steps in the user prompt (early trials)

Either way, I am putting 5-6 steps of instructions in each field, but that’s about the limit of it’s ‘memory’.

1 Like
7

Great advise, if you can get that 5-6 down to 4 or less I typically see a noticeable improvement in deductive reasoning ability for each task. “Time on target” increases and I see better logical deductions. I still wake up thinking non of this is real.

8

Hi @8troy,

Thanks for the suggestions.
How can I scale this approach for my chat bot? I don’t control the user prompts and in my experience, having all the instructions in the system prompt is not helping too. It only follows a few instructions ignores most of them.

Any help is much appreciated!

Thank you!

9

So you can give 10+ instructions if you break them out as separate steps and you make the model repeat each instruction before performing it. GPT-4 can perform 2-3 instructions per line and 3.5 can perform 1-2 per line.

1 Like