I need to send additional data about the user typing to the assistant.
Like E-Mail and User-ID for support purposes and function calling.
I tried using metadata field but it seems the assistant is not using this information.
How could I provide the assistant with this information (in the background)?
Metadata is out-of-band storage of some information that just exists for your utility.
Example metadata takes imagination, like if you wanted to include “user-message-tokens” for some reason, or billing-to-customer-id, as if you could ever establish how much tokens were actually consumed per run or how much you were billed by listing run steps that show nothing about the amount of the thread or retrieval actually passed.
The only thing you can do to augment AI knowledge is either amend the current user message with more information that appears to come from the user, or completely replace the “create assistant” “system instructions” with a new instruction.
Thank you. That is very sad but I found a solution that kinda works for me.
Creating an assistant each time does not work for my case, that would be hundrets of assistants.
Instead when creating the thread, I create it with a starting message containing the users data.
The assistant can later use that message.
You don’t have to make a new assistant to have a new instruction.
You can specify a new instruction when you start a new run.
Makes the whole effort growing more silly at that point, though.
Consider: if you have retrieval, you just paid 50000 tokens to say “my name is Bob”.
You can send the additional information about the user to the Assistants when you start a Run.
const run = await openai.beta.threads.runs.create(
threadId,
{
assistant_id: process.env.OPENAI_ASSISTANT_ID,
instructions: `Please address the user as ${fullName} with email address ${emailAddress} and user id ${userID}.`
}
)
That would replace the assistant’s original programming with only the new text.
“see reference”
Not really. According to the documentation,
You can optionally pass additional instructions to the Assistant while creating the Run but note that these instructions override the default instructions of the Assistant.
It can override if you override your previous instructions. But not if you just append additional info.
You can report back
Assistant instruction: “You are CatGPT, a cat simulator that can only meow and purr”
Run instruction: “my name is Bob”
See if it still meows.
Challenge accepted! wait for a minute…
Edit: You are right! lol that is a bummer… it can’t be true lol I need to test this more.
Edit 2: @_j Okay I made it work. Thanks for the heads up on my misunderstanding. I just retrieved the previous instruction and appended the additional info.
[
{
"id": "msg_hgbOISKBCRGPLo0Qj21kbY2v",
"object": "thread.message",
"created_at": 1700442086,
"thread_id": "thread_p1EVDvZwnPCXNwzCxYXVVXMw",
"role": "assistant",
"content": [
{
"type": "text",
"text": {
"value": "Meow, yes you can! Haha, purrrr!",
"annotations": []
}
}
],
"file_ids": [],
"assistant_id": "asst_MIqYgYXFI9s9TgllyjUawQ73",
"run_id": "run_xS9JbUNqjKdu3eCHOBSMFByt",
"metadata": {}
},
{
"id": "msg_gP8aVyHyPkQga9mbeBag5EAk",
"object": "thread.message",
"created_at": 1700442085,
"thread_id": "thread_p1EVDvZwnPCXNwzCxYXVVXMw",
"role": "user",
"content": [
{
"type": "text",
"text": {
"value": "Yes you can! lol",
"annotations": []
}
}
],
"file_ids": [],
"assistant_id": null,
"run_id": null,
"metadata": {
"id": "1700442085072llj4cgjk1ie7"
}
},
{
"id": "msg_7Xb2KRakeTbQjgH1Q6wuQc22",
"object": "thread.message",
"created_at": 1700442077,
"thread_id": "thread_p1EVDvZwnPCXNwzCxYXVVXMw",
"role": "assistant",
"content": [
{
"type": "text",
"text": {
"value": "Meow, can you fix it? Yes, you can! Purrrr.",
"annotations": []
}
}
],
"file_ids": [],
"assistant_id": "asst_MIqYgYXFI9s9TgllyjUawQ73",
"run_id": "run_XpIhu6QCNqWwif8HzA0GeS58",
"metadata": {}
},
{
"id": "msg_apEsTNL6d5NP0UBdQjLkRvn9",
"object": "thread.message",
"created_at": 1700442076,
"thread_id": "thread_p1EVDvZwnPCXNwzCxYXVVXMw",
"role": "user",
"content": [
{
"type": "text",
"text": {
"value": "Can I hear you say, Can you fix it?!",
"annotations": []
}
}
],
"file_ids": [],
"assistant_id": null,
"run_id": null,
"metadata": {
"id": "1700442075881l8ffjpe48pf9"
}
},
{
"id": "msg_xi4xjkQYCSJredJU3OdaCekH",
"object": "thread.message",
"created_at": 1700442058,
"thread_id": "thread_p1EVDvZwnPCXNwzCxYXVVXMw",
"role": "assistant",
"content": [
{
"type": "text",
"text": {
"value": "Meow, good morning, Bob the Builder! How can I help you today? Purrrr.",
"annotations": []
}
}
],
"file_ids": [],
"assistant_id": "asst_MIqYgYXFI9s9TgllyjUawQ73",
"run_id": "run_Z98TcGljWevwnCQfxxIZsYCC",
"metadata": {}
},
{
"id": "msg_JyWJYprxZWOiQ6iwjFnoVrWm",
"object": "thread.message",
"created_at": 1700442056,
"thread_id": "thread_p1EVDvZwnPCXNwzCxYXVVXMw",
"role": "user",
"content": [
{
"type": "text",
"text": {
"value": "Good morning!",
"annotations": []
}
}
],
"file_ids": [],
"assistant_id": null,
"run_id": null,
"metadata": {
"id": "1700442056530fe93inf9eck"
}
}
]
The issue with that approach is that it will eventually go out of scope as the thread gets longer.
You can either append the data to every user message (what I’ve been doing) or periodically inject a new user message with the data into the thread (every 10-20 messages)
You could modify your assistant. Then have your instructions include variable flags like {email} or something that you could replace with your code. I haven’t personally used it yet as a noob buuuttt seems like it could work.
They don’t support variables in instructions. Would be nice if they did but they don’t. The only way to pass in variables is via a user message and even then you have to be careful…
As an example, let’s say you added the current date to every message and the user has a conversation that lasts 5 days with the highest volume of messages on 11/10/2023. The model is likely to always think it’s 11/10/2023 because it has more example patterns with that date then any other date.
hmmm that kinda sucks and makes sense. You might be able to get around it by completely rewriting the string? just parse out placeholders and replace them with the variables and then pass them back in. Think would have to air on the side of caution here though.
Assistants work well for simple cases but for more complex agents there are much better approaches you can take. You’ll have to do a bit more work but you won’t run into the numerous limitations that assistants have.
As mentioned above… if you change the instructions for an assistant it changes it globally. You’d need to create an assistant per user to use that approach.
Use assistants if your task is simple but look at other agent frameworks if you want more flexibility.
“other agent frameworks” can you recommend? Im new
The two agent frameworks I’ve built are AlphaWave:
And the Teams AI Library:
The Teams AI Library is arguably the more capable of the two (it’s newer and has a feature called Augmentation that’s not in AlphaWave) but it’s focused on the MS Bot Framework. AlphaWave is the more general purpose framework and I’ll be updating it with Augmentation soon.
The Teams AI Library is just a super customized version of AlphaWave
There is a way. Again, using Run. You just need to get your previous instruction and append the additional info. Retrieve Assistant contains instructions field so this is possible even if you created your Assistant from the Playground.
const assistant = await openai.beta.assistants.retrieve(process.env.OPENAI_ASSISTANT_ID)
const assistant_instructions = assistant.instructions
const run = await openai.beta.threads.runs.create(
threadId,
{
assistant_id: process.env.OPENAI_ASSISTANT_ID,
instructions: assistant_instructions + `Please address the user as ${fullName} with email address ${emailAddress} and user id ${userID}.`
}
)
Ill have a read here in a bit. At the moment I am shouting at the screen trying to get my Assistant to talk to me XD.