I have a very long system prompt and set the temperature to 0.8

my observations:

  • with a very long system prompt you need a much higher temperature to avoid overfitting

  • system prompts are not adhered to very well, better to send the instructions in a user prompt and fake the assistant returning and appropriate response.

do you agree?

Depends on the model you are using in my experience. For gpt-3.5-turbo and the like, this sounds correct. For gpt-4 however, it responds really well to how information is passed off into its system message.

The long system prompt overfitting problem is sometimes desirable tbf. When you are generating code or want the output to be in line with the samples that you have provided, you’d prefer a lower temperature. Depends problem to problem

1 Like

I’m a big fan of using long prompts to get more precise content. In my experience, the effect of temperature is manageable. This is because in addition to long and detailed explanatory prompts, I will use multiple levels of prompts to adjust.

1 Like

Hi Pilo. I agree with you. I have found a lot of success with long prompts. Would you minds sharing some ideas?

For me, one thing I do is draw the AI in through descriptions that if the reader was human would be seen as “encouragement.”

I don’t understand all of the mechanics but given the LLM is trained for helping, I open with explaining how my request is going to help me and the AI is the one who will be doing the help.

It feels to me but like calling soldiers to attention in a military setting. A couple of lines of encouragement seems to gear it up for good work, I think slightly better work than cold task descriptions.