I’m kind of new to the assistants API. and how they work. I’m building a meal planner web app where i take the user data communicate with the assistant API i built to generate a An array with the meal plans. However i’m a bit confused about how to manage assistants and threads. From my understanding, Threads holds the conversation context between a user an the assistant. So does it mean:
That i create one assistant in my open api playground and i interact with it for each user data submission? and to differentiate between the users i create a thread for each one of them since this what threads are meant for ?
OR
Duplicate the same assistant config for each user and create a thread for each new conversation?
Please note that except for the meal plan generation there won’t be much interaction with the assistant from the user beside for example asking for meal regeneration for example.
There are several other considerations to keep in mind besides just assistants and threads. However let’s get some of the elementary questions out of the way.
R1. You should create one assistant and use it to interact with the user through one thread per user.
Some Other Key Considerations
Consistency of Assistant’s response
You should not expect that the assistant created out of AssistantAPI will have consistent reponse currently. This is because the beta version of AssistantAPI does not provide access to temparature and top_p needed to make response somewhat deterministic. See here (openairetro/examples/temparature at main · icdev2dev/openairetro · GitHub) for an alternative if consistency is important.
Evolution of Assistant
The assistant is likely not static and it may (should) change with the input from the users. How you intend to do this is upto you. I will have a example of how this could be potentially be done in an upcoming post.
Associating a thread with a user
The thread api does not provide a mechanism to associate threads with users natively. Watch the forum for a potential way of doing this at some scale (assuming that your web-app is successful, how to track 10k users WITHOUT using any other storage mechanism except assistants, threads and messages)
These are some of the consideration, right off the bat.
You could do either. You will need a unique thread per user, but you can do so by either creating a new Assistant with one thread, or use an existing Assistant and generate multiple threads. I think we will see more guidance, either through design documents or through billing/cost policies, from OpenAI on which way to go in the future, but at this point, I don’t think we know which is the “best” way yet.
The pricing model of retrieval data, perpetually billing you daily for every assistant instance to which you attach files, makes it pretty clear:
Assistants should be treated like a custom model, or a unique application
(Although to clarify: they are not anything more than instructions; there is no specific AI set up and waiting just for you)
If you have a multi-user meal planner site, then you only need to craft one “create assistant” with your instructions, documentation, tools. A large number of users and threads can simultaneously interact with it.
As you heard from others above (and they are more experienced in this than I am), there are lots of ways to go about what you want to do. It does take a bit to get your head around the structure of “Assistants”. Basically an Assistant is a persona that you create (via Instructions); it is persistent in nature. A Thread is a conversation with a single user - a chain of messages between that user and one or more Assistants. A Run is where you send a Thread to an Assistant and say “respond to this”; you get back the Thread with a response from that Assistant (and you can tweak the Assistant’s persona with additional_instructions for that Run, which is one way of having just one Assistant but still having it behave differently when you need it to). It is a lot to get your head around but once it falls into place, it all makes sense.
That said, i think that you might want to consider scaling the Assistant structure as well.
For example, there’s the “Keto” diet, the “Vegan” diet, the “Satvic” diet etc. Each one of the Assistants could be “callable” from the main Assistant (obviously only one of them).
And in that context, scaling likely means NOT storing docs for each assistant.
As for point 4, ik that I won’t be able to associate threads with users natively. I’m planning to do that by storing each user’s thread in the db. Since the assistant will hold the context for every thread.