Assistants API - Access to multiple assistants

Is there a way to have access to multiple assistants in the same thread? I want to be able to choose an assistant based on the context of the conversation.

6 Likes

You might still have more questions not answered yet - like if this does exactly what you think it might do.

1 Like

I wanted the model itself to dynamically switch between one assistant to another depending on the prompt, as opposed to updating it manually using the API.

The threads are managed separately from the runs. When you create a run you give it a thread id and an agent id. When you create the run, it’s a one-time thing, like a chat completion request … It will either append a message to the thread and then go to completed state or else go to requires action state for you to supply function results and then it’ll append a message and go to completed, see here. So, you create a fresh run, supplying assistant and thread id, each time you want an assistant response. And for user input you just add a message to the thread yourself. There’s no reason you couldn’t get a message from the user and add it to the thread, decide (e.g. through a secondary chat completion of assistant call) what assistant to use to answer it, create run with that assistant and have it add its answer (possibly requiring you to supply function call results first), get new messages from user, decide to use different assistant etc, and have them working on the same thread. You could dont even have to wait for user input, you could create two runs in a row with different assistants. I don’t believe the actual chat completion calls the Run does in the background include assistant ids, so it’ll appear to the the assistant current answering as if it supplied all previous assistant answers, which could throw it off to some degree e.g. if the assistant write in very different styles … But otherwise it’ll work just fine.

4 Likes

I had a similar question.

I’ve built crude “roundtables” of AI councils but the idea of having Assistants with a specific set of skills like Liam Neeson?

Unreal.

Need that!

2 Likes

“Depending on the prompt” means another AI has to first classify the best place to send a conversation and its context off to.

Like I wrote a classifier to pick a temperature for the language model to be called with.

This may or may not be what you are looking for. I have two assistants using the same thread. One is a fiction writer the other is a critic. The writer creates a first draft of chapter 1, then has the critic provide feedback, the writer then rewrites the first draft. It may not be what you are looking for, but perhaps it moves you closer to what you are after.

import time

from openai import OpenAI

# gets API Key from environment variable OPENAI_API_KEY
client = OpenAI(
    api_key="your API key",
)

assistantWriter = client.beta.assistants.create(
    name="Writer",
    instructions="You are an expert writer of fictional stories",
    model="gpt-4-1106-preview",
)
assistantCritic = client.beta.assistants.create(
    name="Critic",
    instructions="You are an expert critic of fictional stories. You provide positive and constructive feedback",
    model="gpt-4-1106-preview",
)

thread = client.beta.threads.create()

message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="""Write a single chapter about a young girl that meets a centaur in the forest. 
    Describe how she feels, what she sees, hears and even smells""",
)


def runAssistant(assistant_id,thread_id,user_instructions):
    run = client.beta.threads.runs.create(
        thread_id=thread_id,
        assistant_id=assistant_id,
        instructions=user_instructions,
    )

    while True:
        run = client.beta.threads.runs.retrieve(thread_id=thread.id, run_id=run.id)

        if run.status == "completed":
            print("This run has completed!"

            break
        else:
            print("in progress...")
            time.sleep(5)
                  
# Run the Writer Assistant to create a first draft                      
runAssistant(assistantWriter.id,thread.id,"Write the first chapter")
                  
# Run the Critic Assistant to provide feedback 
runAssistant(assistantCritic.id,thread.id,"""Provide constructive feedback to what 
the Writer assistant has written""")
                  
# Have the Writer Assistant rewrite the first chapter based on the feedback from the Critic        
runAssistant(assistantWriter.id,thread.id,"""Using the feedback from the Critic Assistant 
rewrite the first chapter""")

# Show the final results 

messages = client.beta.threads.messages.list(
  thread_id=thread.id
)

for thread_message in messages.data:
    # Iterate over the 'content' attribute of the ThreadMessage, which is a list
    for content_item in thread_message.content:
        # Assuming content_item is a MessageContentText object with a 'text' attribute
        # and that 'text' has a 'value' attribute, print it
        print(content_item.text.value) or paste code here
17 Likes

Is there any advantage to creating the assistants in the playground vs in code?

1 Like

I’d say advantages of code are:

  1. Version control
  2. Harder to make mistakes
  3. Easier to scale
  4. Easier to add tools/functions.

Aside from that though they re doing the same thing. We still create them using the Playground as it’s quicker for us and don’t need to create many!

1 Like

An approach you may want to consider is the addition of a “Facilitator” assistant. The assistant:

  1. Has clear definition of roles and expertise of each other assistant in rhe ‘group chat’.
  2. After a message is added to the thread (multiple assistants can run on the same thread) the “Facilitator” is run.
  3. The “Facilitator” reads the last message added, notes the message_id, and determines which assistant should respond. A message is added to the thread that includes the message_id and respondent assistant.
  4. Code calls a run of the respondent assistant to read message_id, and respond.
  5. The loop begiins again when the Facilitator read the last message posted.

The downside of the approach is the Facilitator may become a performance bottleneck, if the thread activity is high.

3 Likes

this is interesting. but i believe the ideal scenario is having the 2 assistants assigned to the same thread (something like assistant_id = [assistantWriter, assistantCritic]) and the user role guides their actions via the messages.create endpoint. (not through the run instructions).

there’s a mention of this in the Assistant API notebook in the cookbook but it isn’t linked to anywhere:

There’s a few sections we didn’t cover for the sake of brevity, so here’s a few resources to explore further:

  • Annotations: parsing file citations
  • Files: Thread scoped vs Assistant scoped
  • Parallel Function Calls: calling multiple tools in a single Step
  • Multi-Assistant Thread Runs: single Thread with Messages from multiple Assistants
  • Streaming: coming soon!

Now go off and build something ama[zing]

2 Likes

Hey, this code is very helpful thanks. I have a question if you dont mind. Sorry bout the light weight necro

Whats with the message and the run, the message asks a big question the run asks a subset? It just seems like a weird extra step and redundant. What if I just want to ask 1 question, like how do I sort a list in python? Would I do like instructions=“Your a professional developer” then thread =“You answer code questions” and then in the run “How do I sort a list in python” ??

Or maybe better, is thread supposed to be a more general version of the questions asked in run?

This diagram may be of some use. This corresponds to the code I shared earlier. The objective of this code was to provide a simple example of how to use the Assistants API where two kinds of assistants interact with each other using a single thread. I am learning this as I go, but this is my current understanding. Not everything you do needs the assistants API, but having two AI experts interacting with a human user is one of them.

2 Likes

Can you say anything about the quality of the result doing it like this? I imagine that both assistants bleed into each other, as when they run, they both think that the whole conversation was created by them and that no other assistant in this thread exists. But maybe this has no influence actually on the result.

1 Like

It’s difficult to answer that. What I do know is that I can see signs that the writer assistant is following the constructive feedback of the critic, but whether that leads to a “better” story is subjective. Note, for example that the critic after reading the first draft suggests making the Centaur’s dialogue sound less modern, which the writer picks up for the second draft. Here is a sample of the output. Fun to read, at any rate.

User message:
“Write a single chapter about a young girl that meets a centaur in the forest. Describe how she feels, what she sees, hears and even smells.”

Writer: Chapter 1: The Meeting in the Verdant Thicket

Every village has its secrets and tales, woven into the fabric of its existence like threads of gold in a grand tapestry. For the small hamlet of Eldenwood, nestled on the fringe of an ancient forest that seemed to stretch indefinitely into the mists of the unknown, the legends centered on the denizens of the deep woods.

Elnora, a young girl of no more than twelve summers, had always been enchanted by these stories. Her heart danced with curiosity and her mind teemed with the sort of fearless wonder that only the innocence of youth could sustain. With her auburn hair tied back loosely and her dress, a gentle swirl of greens and browns that mirrored the earth tones of the forest, she stepped lightly between ferns and fallen logs, deeper into the domain of whispered legends.

The air was rich with the scent of damp earth and the verdant aroma of lush foliage, freshly awakened by the recent rains. She took in deep breaths, the cool air playing upon her lips, as the quiet symphony of the forest greeted her—birdsong from the canopies, the low buzzing of insects, and the occasional rustle of a small creature foraging in the undergrowth.

Just as she reached a sun-dappled glade, there came a soft thudding, punctuated by a cadence that her ears could not quite recognize. This was no familiar patter of a deer’s hoof or the scamper of a squirrel. Her heart quickened, thumping against her chest as if in rhythm with the strange gait.

And then she saw it—a creature as majestic as it was surreal—the figure of a centaur emerged from the dappled shadows of the trees.

Elnora’s breath hitched, and she hid behind a broad oak trunk, peering out with wide, unblinking eyes. There before her stood a being of such profound harmony between man and horse that she wondered if she had tumbled into the realm of dreams. His coat, a lustrous chestnut, glinted where the sunlight kissed it, and his human torso was that of a young man with sun-bronzed skin, strong and lean, with hair like spilled ink that cascaded to his shoulders.

The centaur’s ears twitched, alert to every rustle and whisper of the forest, his deep brown eyes scanning his surroundings with an air of dignified caution. Elnora’s heart danced with a mingled thrill of excitement and fear—creatures like this were said to possess wisdom and strength beyond that of mortal men. Yet, her village had also spun tales of their fierceness in battle and their unyielding protectiveness of the woods they called home.

The scent of woodsy musk drifted to her, a wild spice that reminded her of autumnal leaves and the free, untamed spirit of the forest. Elnora took a step back, her foot dislodging a stone, which scraped and snapped a twig beneath.

The centaur’s head whipped around, his gaze piercing the bramble and underbrush until his eyes found hers. She was rooted to the spot, the world seemingly frozen at the junction of their locked gazes.

He could have charged or bellowed for her to leave; instead, he took a step forward, with a grace that belied his size. His voice, when he spoke, was deep and melodic, echoing the natural symphony around them.

“Why do you enter these woods, child of man?” he asked, his words tinged with an old-world cadence that seemed to carry the very whispers of the forest.

Elnora swallowed hard, finding her courage and her voice. “I—I came for the stories,” she stammered, her inherent trust and fascination with the creatures of legend pushing her fear aside. “To see if the tales were true.”

“And you have found your story,” the centaur stated, a gentle hint of humor softening his tone. “Though perhaps you did not expect it to be quite so tall.”

Despite the gravity of this encounter, a smile found its way onto Elnora’s face—a reflection of the pure joy that the discovery of this wonder brought her. She stepped out from her hiding place, still cautious, but driven by her unquenched curiosity.

The centaur regarded her with an appraising tilt of his head, sensing the purity of her intentions. And in that moment, an unspoken agreement was forged beneath the ancient boughs of Eldenwood—a pact of mutual respect and a shared secret to be cherished.

As the sun gradually journeyed across the sky, casting shifting patterns of light and shadow upon the earth, the centaur began to share the untold stories of the forest. The young girl listened, rapt with awe, every sense alive to the magic of this place and the incredible companion she had met within its heart.

Her fear all but forgotten, Elnora realized that the forest was more than just root and branch—it was a living narrative, and she had become a part of its endless story.

Critic: The chapter you’ve written paints a vivid and enchanting encounter between Elnora and the centaur, providing a well-balanced mix of descriptions and narrative that brings the scene to life. Here are some constructive points of feedback:

  1. Pacing: The pacing in the first half of the chapter builds a good sense of anticipation as Elnora enters the forest. However, once the centaur makes an appearance, you might want to slow down the narrative a little more to extend the gravity of the moment—the first eye contact between the girl and the mythic creature.

  2. Sensory Descriptions: You’ve done well to include various sensory descriptions—particularly the scents and sounds of the forest—which help immerse the reader in the setting. To enhance this further, you might consider describing the texture of the tree’s bark that Elnora hides behind, the sensation of the forest floor beneath her feet, or the physical reactions she might have to her fear and excitement (e.g., goosebumps or the feel of her own breath quickening).

  3. Dialogue: The centaur’s dialogue is appropriately otherworldly and contributes to the character’s mystique. Be careful, though, to ensure that Elnora’s response doesn’t seem overly modern in comparison, which could break the story’s enchanting spell. Try to give her language a timeless quality that matches the centaur’s tone.

  4. Character Development: While the initial exchange is well-handled, further development of Elnora’s feelings during this interaction would be beneficial. Is she overwhelmed, relieved, captivated, or all of the above? A deeper dive into her inner turmoil would create a stronger connection between the girl and the reader.

  5. Centaur’s Introduction: When you introduce the centaur, the description is effective, but the text may benefit from a more dynamic action or demonstration of his power or mystical nature. This would avoid him appearing too passive or human-like when he first comes into view.

  6. The Ending: The conclusion of the chapter is touching and it effectively introduces the promise of more stories to come. However, it could be made clearer what the ‘unspoken agreement’ between Elnora and the centaur entails. Is she now a protector of the forest’s secrets, or is there a different expectation set upon her?

Remember that each chapter should serve to push the story forward, develop the characters, or deepen the world-building. This chapter accomplishes these goals well, but careful attention to pacing, consistent dialogue, and further exploration of the protagonist’s emotions will enrich the narrative even more.

Writer: Chapter 1: Enchantment in the Eldwood

In the hamlet of Eldenwood, where the whispering forest skirts the edge of reality, there existed tales of beings that dwelled within the reach of the ancient trees. Among them spun the legend of the centaurs, the wise keepers of the woods with the bodies of horses and the torsos of men.

Elnora, a girl of only twelve summers, had always felt the pull of these stories. A voracious imagination fueled her courage as she wandered through the forest with a heart buoyed by wonder. Auburn locks were wrestled into a casual braid, while her dress, a patchwork mimicry of the forest’s hues, swished with each intrepid step.

The air was a fresh caress, fragrant with the musk of rain-kissed earth and greenery in fulsome bloom. Elnora inhaled deeply, savoring the tang of pine and the loamy undertones that danced upon her palate. The soundscape of the forest enveloped her—an orchestrated serenade of chirping birds, hidden critters scuttling in the brush, and the rustle of leaves stirred by a gentle zephyr.

Amidst this tranquility, a series of thuds, unlike any forest cadence she knew, caused her heart to cease its rhythm in a moment of startled curiosity. It was a sound that defied her understanding—an exquisite blending of the familiar and the bizarre.

Through the bracken and brambles, into the clearing she stepped, and there it stood—a centaur, undeniable and resplendent in the dappling light that filtered through the leafy canopy.

Elnora’s breath became a startled gasp that clung silently in her throat. Her instinct was to retreat to the shelter of an ancient oak that stood as guardian of the glade. Wide-eyed and ensconced in shadow, she glimpsed the figure before her, an embodiment of myth made flesh.

His equine form shimmered with a coat of rich chestnut, burnished to a shine where sunbeams touched. Above this noble beastly grace, the centaur bore the torso of a young man, sun-kissed and sinewy, with a mane of jet as wild as the woodlands from which he emerged.

Alert to her presence, the centaur swiveled with an elegance that betrayed his imposing form, his deep eyes, pools of earthen wisdom, meeting Elnora’s own. From his lofty height, he studied her—not with malice, but a discerning calm that bade her to stand her ground.

A new scent wove itself into the air—the heady mix of untamed nature and something indefinably other, a whisper of sage and wildflowers that seemed to emanate from his very presence.

Movement betrayed her—a shift of footing that sent pebbles tumbling, a snapping twig that shattered the silence. Their gaze was locked, a bridge across the void of apprehension.

“Child of man,” his voice resonated, each word a melodic intonation echoing the natural resonance around them, “What brings thee to these sacred bounds?”

Elnora, caught between the tales of valor and her own sprouting boldness, found her voice. “I came seeking the truth behind the stories,” she replied, her words dipped in the honesty of her quest.

“Then it seems thee hast found thy pursuit,” he uttered with a softness in his tone that surprised her—a jovial undercurrent that felt like a smile shared by old friends.

Her fear ebbed away, and in its place bloomed a joyous daring. She stepped forth from her wooden sentinel, the cool shade relinquishing its hold. “I did not expect it to be so… real,” she admitted, a smile finding its own truth.

With a nod, the centaur acknowledged the brave girl. A connection formed, silent and steadfast, beneath the watchful trees of Eldenwood—a pact of wonder and secrecy kindled between two souls from worlds that seldom met.

Elnora listened, rapt and spirited, as the centaur began to unfurl the stories that the forest guarded—tales of enchantment, of ages old and new. She was entranced, wholly immersed in the centaur’s world, every sense alight with the magic that swirled in the clearing where legend and child had collided.

Here in the forest, a place beyond mere root and branch, Elnora found herself entwined in a story much grander than she had ever hoped—and there, amongst the whispering leaves, a young girl’s belief in the wondrous was affirmed.

2 Likes

The big question is then: Are the assistants in one single thread assistant-aware? So that they use the info from other assistants as it’s the same thread, but work on their instruction, not bleeding into each other?

I don’t expect an answer from you, I’m just wondering what the concept here might be from the developers of this API.

1 Like

Maybe the developers of the API haven’t figured this out for a while either, and until they do, giving the most freedom for users to build their own may reveal the most efficient way to do this, and waiting for inspiration takes a little while.

1 Like

@yangdafu123 yeah, it’s super nice to explore these things.

this is what i was wondering also, if the info is shared, but parceled into the different roles and functions for each assitant to take up. I suppose there would be the main assistant who discerns where each task would go.

If you precede each assistants’ comments with their name (or title), as dlaytonj2 does in the example… preceding each assistant’s response with “Writer:” and “Critic:”… and you let the AI’s know that everyone in the chat thread will “have their name preceding their comments”, then it works. I haven’t tested it heavily, but my initial tests indicate the AI’s can keep their comments segregated by name. Not sure if it eventually loses track or hallucinates. I’ll try to come back and let you guys know after more testing.

1 Like