System messages in Assistant threads

API Feedback
,
1

Assistants beta V2 seems to be considerably better than V1. I abandoned V1 due to poor performance, unreliability and lack of streaming.

With V2 now in place, I’m tempted to switch back to assistants, but there is one thing missing for my use cases: the ability to add system messages to threads. The Assistants API restricts the message types to “user” and “assistant”. I realise that “instructions” can be passed to the API (which I guess become system messages under the hood) but I have use cases where I need add system messages at specific points in the thread.

Will we ever have the ability to add system messages to assistant threads?

2

When you run the assistant.id in the thread.id, will run instructions automatically in the first run.

4

@razvan.i.savin I know that. I even say so in my question.

6

It seems like you are echoing AI responses. I’m less interested in the current state of affairs compared to future plans.

1 Like
8

I have the same issue. When we use Function Calling in Assistant APIs using 3.5 Turbo, the whole models hallucinates around the parameters. In Chat Completion API we can pass the System Context to fix this but in Assistant API, we only have User and Assistant Roles, which makes no sense.

This thing makes Function Calling using 3.5 literally worthless. Also, I don’t know what developers are so worries about the System Context in Assistant when it is simple available in Chat Completion.

Currently there’s no way of passing the System Context at certain point in conversation in Assistant APIs, which makes no sense at all.

1 Like
9

I’m facing the same issue. I need to add context to a thread at creation time. The assistant’s instructions can be good for generalized context, but i need to add user specific context for a thread. System messages in Chat Completion has this but I wonder why Assistants does not !!??

Have you found any solutions @MrFriday @adamralph

10

Not yet, this one single issue make GPT 3.5 Completely useless. But if you understand the concept of Function Calling, you can fix it to some context.

11

At what point do you need it?

Currently, there is the additional_instructions parameter when you create a Run. This will append additional instructions at that run only.

For example, when you send a new message, you add this “system message” using the additional_instructions parameter to the run. You can change it the next time you send a new message.

The practical usage for this is when you want to append user info, order details or datetime for function calling reference.

2 Likes
12

No. I’m using the regular completions API instead of the assistants API.

13

For this use case, the solution suggested by @supershaneski may be enough:

1 Like
14

Thanks for the post @adamralph and all of the replies.

It seems like @supershaneski has a solution with “additional_instructions” but I think some of the spirit of the original question is unanswered and I’m still curious about that if anyone can provide tips. I’m wondering: to what extent can we configure the basic “system” in Assistants? I mean more like built-in API config options (less like additive techniques such as RAG, fine-tuning, etc).

It seems the answer is: we get to set the initial description/instructions then use additional_instructions at points after that.

15

I gave up on all the wonderful promises of assistants api and have implemented a chat completion pipeline where i pull up context messages as Pagable objects in java-spring and call the embedding api for RAG, works fine.

Having said that, in an ideal world if assistants api was able to fix this prob, it would simplify my codebase:

  1. I don’t have to worry about persisting chat messages to DB and pulling them up every time I make an api call (Threads would solve this)
  2. Have RAG out of the box pipeline as file search handles this

this would be highly effect in apps like mine where i only care about the end result of the chat i.e. the parms that the model returns as a function call back.

16

Thanks ! I’m going to test this outside my app, and see if this works as expected. Will post the results here.

17

Note that in the Run, you have two parameters you can use:

If you want to just “add” to the original instruction/system_prompt, use the additional_instruction.
If however you want to alter the original instruction, use the instructions parameter. This will replace the instruction.

4 Likes
18

I think this solves my problem. Thanks man !

1 Like
19

+1 thanks all for the explorations here.

20

Thanks guys for all the knowledge, @supershaneski really killed it with his answer about additional_instructions.

Would you guys happen to know if the additional instructions are only considered in the run they are present? Or does the model have this in the thread for subsequent runs?

21

Additional_instructions are a run parameter. They are treated like part of an assistants’ operation, adding to its internal system message for that run.

While they may affect an AI - and the past responses may confuse an AI if they are later changed or removed, the system messages are not part of a thread’s history of conversation.

1 Like