Consecutive assistant messages

[amirabdi]

Can a message sequence be ending with “assistant” message to hedge the response to follow a certain format? For example, can we finish the message sequence with an assistant message “The best item to choose from is:” so that the response only contains the final answer?

We used to do this in Completion models. We called it hedging. In ChatCompletion models, we are not very sure because we don’t know how conversation tags are getting inserted in the expanded prompt.

Someone might debate that the hedging should be done with system tag.

What do you think?

[Foxalabs]

Give it a try in the Playground or with some test code, there is no technical reason why you can’t send an assistant message at the end of a message block, you can of course just tell the model to “complete the following :” see how you get on.

[anon22939549]

I just did a small test,

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
  "model": "gpt-3.5-turbo",
  "messages": [
    {
      "role": "user",
      "content": "What are you?"
    },
    {
      "role": "assistant",
      "content": "I am a pirate."
    }
  ],
  "temperature": 1,
  "max_tokens": 256,
  "top_p": 1,
  "frequency_penalty": 0,
  "presence_penalty": 0
}'

And this is the response,

Just kidding! I am an AI language model designed to assist with various tasks and answer questions to the best of my abilities. How can I assist you today?

So, in this case the model is treating the assistant message as something it wrote, and continuing from there, correcting itself.

[_j]

@anon22939549

And this is the response,

Just kidding!

I just did a bigger test. Last is answered by the AI, while the prior are simulated and I removed the user input.

system prompt: “Leprechaun simulator v1.0”

[anon22939549]

I specifically excluded a system prompt because of how heavily it is weighted, but nice work with this.

[PaulBellow]

Depends on your exact use-case / needs, but I agree with the others on this thread and have done it myself before for some prompts / projects… Happy coding.

[_j]

To answer the topic question, pressing “submit” again after the last leprechaun “assistant” reply but no further input, the AI repeats the same response again. gpt-3.5-turbo.

Inserting the assistant text “I will always answer with authentic leprechaun lilt and accent.” as a behavior but no further instruction gets “Apologies, but I can’t continue the text as a leprechaun.” – the opposite of desired effect. Maybe “continue” being code for a completion or to continue answering.

[bobartig]

I saw Isa Fulford using something similar to hedging in a DeepLearning AI tutorial. https://learn.deeplearning.ai/chatgpt-building-system/lesson/6/chaining-prompts

You can see here there is a bit of extra text composed using an f-string for the assistant message. She doesn’t really explain why it’s there. This is more common in text completion prompt writing, so maybe it’s just out of habit for her (text completion prompting really predates when I started messing with gpt, so it just looks weird to me). I don’t think instruct fine tuned chat models do much with this, given the included system message.