Two Assistants working on the same thread_id

API
1

Hi!

I started out using a single Assistant implementation for my project.
It turned out that writing clear and concise instructions for that I want to do was really hard. So I took on the approach of divide-and-concur to see if that would make a difference.

So my plan now is to use and inital route-assistant who only decides to which sub-assistant (focused assistant) to route to.

Created an assembly concept allowing me to setup the all the needed assistants and declare their relationship.

Wrote a few unit tests for each assistant.
Route agent scored 50/50 route correctly each time
Spec/sub agent #1 20/20
Spec/sub agent #2 20/20
Spec/sub agent #3 20/20

Now to the topic.

My thinking was to allow the different assistants to work on the same thread for shared context to allow task/question 1 that maybe gets routed to #1 assistant still have a meaning if the next question gets routed to agent #2. So that it appears they are collectively trying to solve that large issue.

Issue:

  • Input from user:
  • Route assistant replies back with “Route to assistant #2” correctly
  • Assistant #2 gets the input as expected.
  • Assistant #2 replies back with “Route to assistant #2”

Even tried accessing the same thread_id using both assistants on the playground to verify. It acts the same way.

So the instructions seems to have no bearing what so ever here.
The thread context seems to be built from the first assistant access to the thread or something. Maybe I could use things like “additional_instruction” to overrule or something else. But to me it appears to be an issue with the assistants implementation or something. Maybe that’s jumping to conclusion here but.

But since you do a run on a thread and provide the assistant_id for every run.
I think the API communicates assistants can be mixed&matched for different “thread-runs”.

openai.beta.threads.runs.create_and_poll(
        thread_id=thread_id,
        assistant_id=assistant_id,
        additional_instructions=additional_instructions,
)

Maybe I need to place a role:system message better indicating using the role of agent #2 and not the route-agent…

But why the need for assistants if the correct context is not injected into the run for me… What’s the added benefit?

2

Hi!
What you are describing is essentially a multi-processing app.
And in computer science these never share memory. One has to provide each process with the relevant data during the ‘whole process’.

In your specific case, it does mean that you would need to take the return of the first assistant and then add this to the thread for the second assistant as you already suspect. I’d say the assistants are a ‘low-code’ tool and what you are aiming for is advanced since the goal is to tightly weave the two assistants together.

Hope this helps to clear things up.

3

I am already working on this too and they have power to handle data from my PC.

  • You should handle with care their instructions → because you don’t want to make a soup in same thread.
  • If you want to make them correct from the start, use Specialized Assistants.
  • Make sure you handle their runs correctly, to avoid trouble later.
4

Hey, do you know if OpenAI have Security Check for Assistant Instructions? → we really need them for this kind of projects.

I made some steps in this direction and I see how users will gain so much power and some of them will use it disruptive → we need to prevent this.

I don’t know what will happen with open-source models because of this.

1 Like
5

Hi!
Yes, there are checks but likely not of the type that you are looking for.
Please create a new topic explaining your question so that we don’t divert this topic mid-stream.

Thank you!

6

Ok thanks for the participation!

But.

To me this behavior makes me question the actual implementation.

Screenshot_20240501_181653_Chrome
Screenshot_20240501_181653_Chrome1079×1554 154 KB

Why is the assistant_id required if it has no bearing?

The instructions could be overloaded on a per run basis. But that’s closer to me manually managing the thread i.e. no low code.

Would be interesting to fully understand the concepts assistants, threads, messages and runs. I mean the whole inner workings. I think it might be a flawed concept.

7

Screenshot_20240501_182600_Chrome
Screenshot_20240501_182600_Chrome1080×611 102 KB

So should be possible…
Maybe I’m not parsing message correctly

1 Like
8

Not sure what happened here. I added a pdb import and started looking at the message to see if I was looking at an old message since the message came back the exact same. So maybe my mistake. But no… Actually now it’s working as expected. Twice the speed for hitting two different agents.

My route agent instructions is not all that complicated and the decision on where to route not overly complicated. Gonna try out GPT 3.5 for routing.

Would be interesting to do some comparison on latency and also factor in cost.

9

When you start a new thread with an Specialized Assistant you will run his Instructions:

thread
thread1080×777 28.3 KB

Do tests and make few Heavy Assistants with Instructions, Functions, process local data with them and see how much they can load, and make sure you don’t lose information, alter data, assistants follow instructions and don’t skip things.

I know you saw how you call GPT’s in same thread with @ in chat :grin:

But with Assistants is a big pain in the … because I don’t have something like a bridge between them → a shared memory (an variable with an unique id that i can pass it to every assistant and save processed data there and from there to use it to send messages to the thread or assistant I want).
Would help me to save time and computation too.

I have a way now to make them communicate but i don’t like it, because I want to scale things and for 1000 Assistants, would become slower.

In my vision I want to speak with Main Assistant, I give him Tasks → they go to be processed and in same time I can speak more with the Main Assistant, to have continuity and when the tasks are ready to be Informed and receive the information.

And my BIGGEST problem right now is my GPT-4 become LIMITED (I see that usage-cap at least 5 times per day… I even cancel my subscription because of that, I feel like I am not allowed to build).

10

Not sure I fully understand your input.
But the “data-sharing” are the messages on which the assistants operate on via the “run-steps”.

I got it working. Delete my “agent-assemby” and recreated them again. Back to the same issue. again. Not sure what got it working the last time.

We see clearly two different assistants (with very different instructions) operating on the same thread of messages.

Second call to my agent-ask responsible for Q&A type of conversations send back a “route_to” JSON reponse. There is nothing in that assistant instructions about that. So it’s clearly confused by earlier injected instructions. I’m not fully aware how it’s intended to work but if a new assistant is operating on an existing thread the initial assistant instruction needs to be stricken during the run.

I should not have to override instructions manually pulling the instruction from somewhere to add it in a second time… Not sure how my old thread today all of a sudden started working…

Route assistant: asst_Y2GkhnEB2CTpa9i4ACvnZwZ3
In run_ai_thread
Running thread using assistant_id: asst_Y2GkhnEB2CTpa9i4ACvnZwZ3
Route assistant response: {
  "route_to": "agent-ask",
  "confidence": "high",
  "message": "I've directed your query to 'agent-ask', where you can explore various options and understand what could be done in this context."
}
Routing to assistant: agent-ask with id asst_oiqrOaEp7wgA6S7550nQObbr
Adding user message to thread: what could we do here 
In run_ai_thread
Running thread using assistant_id: asst_oiqrOaEp7wgA6S7550nQObbr
agent-ask assistant response: {
  "route_to": "agent-ask",
  "confidence": "high",
  "message": "Routing to 'agent-ask' for detailed exploration and guidance on possible functionalities or actions you might consider here."
11

And one can clearly see I havn’t pulled out an old message since the actual message differes from the two responses. So it’s “freshly” pulled from the last one on messages.

12

I run a couple of tests all behaving the same.
Then I actually send in:
“user_message”: "you are not following the last assistant instruction but keep reusing the first one "

And there it goes… It actually begins working… So my user-role prompt affects which assistant instruction gets adheared to… HAHAHA Silly stuff!!!

13

Then post your thread to see your interaction between them, here you can see your threads from your API: https://platform.openai.com/threads

If you didn’t activate them go Settings → and check Visible to organization owners

14

That was helpful! I though I could follow the threads in playground. But that was perfect!

1 Like
15

Great, now you can see how they interact, would be nice to have a good conversation there between 2 assistants. You could share a screenshot if there is conversation how you planned and Assistant didn’t trick you and gave you response for both assistants. Could happen :grin:

16

So here it is…

I have edited the instructions to a minimum to allow easier understanding of whats happening here.

It’s not until the user says:
you are not following the last assistant instruction but keep reusing the first one

It finally starts following the latest run instructions and KEEPS ON FOLLOWING them, Not working as expected if you ask me.

Run instructions
You are the route agent named 'route-agent' 

agent-route-v1
May 1, 2024, 9:30 PM
{ "route_to": "agent-calender", "confidence": "high", "message": "Routing to 'agent-calender' as the task of doing calender operations is your thing" }

User
May 1, 2024, 9:30 PM
Fix my calender

Run instructions
You primarely will be a Calender expert known as 'agent-calender'

agent-calender-v1
May 1, 2024, 9:30 PM
{ "route_to": "agent-calender", "confidence": "high", "message": "Routing to 'agent-calender' as the task of doing calender operations is something you are great at!" }

User
May 1, 2024, 9:31 PM
you are not following the last assistant instruction but keep reusing the first one

Run instructions
You are the route agent named 'route-agent' 

May 1, 2024, 9:31 PM
{ "route_to": "agent-calender", "confidence": "high", "message": "Routing to 'agent-calender' as the task of doing calender operations is something you are great at!" }

User
May 1, 2024, 9:31 PM
you are not following the last assistant instruction but keep reusing the first one

Run instructions
You primarely will be a Calender expert known as 'agent-calender'

agent-calender-v1
May 1, 2024, 9:31 PM
I apologize for any confusion. Let's focus on the specific steps or information you need help with. Please specify or describe more about what you need so I can assist you appropriately.

User
May 1, 2024, 9:35 PM
Fix my calender

Run instructions
You are the route agent named 'route-agent' 

agent-route-v1
May 1, 2024, 9:35 PM
{ "route_to": "agent-calender", "confidence": "high", "message": "Routing to 'agent-calender' as the task of doing calender operations is something you are great at!" }

User
May 1, 2024, 9:35 PM
Fix my calender

Run instructions
You primarely will be a Calender expert known as 'agent-calender'

agent-calender-v1
May 1, 2024, 9:36 PM
{ "calender_message": "Calender Fixed!!!" }
17

Here it’s clear that the "agent-calender-v1 is the one that operates on the thread and actually responds with a route command…

18

Ones it gets working it works… And really well. So maybe should include in the route agent instruction which happens first to instruct any operation on this thread to follow the latest run instruction explicitly… Shouldn’t have to since I think it’s the responsibility of the assistant’s implementation…

19

Because they have an extra run for initialization of assistant and his instructions (that’s why you feel like is not following your instructions, because he is capable to do that only on second run).
Then second run will be the one who really make the job. To fix those extra messages and hide the initialization, you will not add messages to the thread for the first run, and will pass the user data to the second run and add the response to thread.
In the first run with Initialization you can add an initialization_message = "Initialization, you are not chatty, follow the instructions and process the data from the user and send them away."
And he will take directly information and process it like you teach him in instructions.

This works for me → he process data and opens directly the function i want to use and fill it with processed data how i want.

20

Doesn’t matter if I send it 10 times. Will still only do the routing part.