Tip: getting gpt-3.5-turbo to follow instructions

API
1

I’ve been playing with gpt-3.5-turbo all weekend and while it’s impressive, as the docs say, it’s not as good at following instructions as text-davinci-003. I finally had a bit of a break through this morning and I thought I’d share… If you just forget the system message all together, and instead provide your entire prompt as an extra initial user message it seems to do much better.

I tried splitting some of the prompt logic across the system and initial user message and that just seemed to confuse it. Best results are when I’m not passing a system message at all. At least in the playground… Hope that helps others…

5 Likes
2

Hi,
Can you be more specific about what user and system messages are?
I’m encountering the same problem of gpt-3.5-turbo not taking its instructions well, too…
It would be nice if you provide some more explaination, even some examples!
Thank you!

3

It’s likely he’s referring to the structure of a ChatCompletion run:

openai.ChatCompletion.create(
messages = [
{‘role’: ‘system’, ‘content’:
“”“You answer ‘Yes’ or ‘No’ to the question.
“””},
{‘role’: ‘user’, ‘content’: <your_query>}
],
model = GPT_MODEL,
temperature = 0
)

4

The post was made just a few days after the release of the Chat API, which takes an entirely different style of instructing than the previous completion and instruct models, a learning curve with an influx of questions and support.

I can also complain about a newfound ability of gpt-3.5-turbo to simply not understand semantically-clear specifications and instructions, but that would be taking the topic off-course, where a solution for the initial concern was likely implemented months ago.

1 Like