I am trying to integrate Assistant API with my application, which involves a conversation between AI and the player. However, due to some business logic requirement, I need to make sure the AI only return one message after the player gives one message. Right now, sometimes, the AI will randomly return multiple messages at one shot in the response… I don’t know if there is any parameters or any way that we can restrict that…
Hi and welcome to the Developer Forum!
Can you give some examples of this and possibly a code snippet of your APi calls?
cant you limit you messages ?
const messages = await this.openai.beta.threads.messages.list(
run.thread_id,
{
limit: 1,
order: ‘desc’,
},
);
1 Like
Hi sure:
export async function GET(request: NextRequest) {
const { searchParams } = new URL(request.url);
const threadID = searchParams.get('thread_id');
if (threadID == null) {
return new Response('Missing thread_id', { status: 400 });
}
try {
const messages = await openai.beta.threads.messages.list(threadID);
return new Response(JSON.stringify(messages.data), {
headers: { 'Content-Type': 'application/json' },
});
} catch (error) {
console.log(error);
return new Response('Error getting messages', { status: 500 });
}
}
My problem is the messages I get will sometimes contain multiple response from the AI in multiple message object. I don’t want to do a frontend hard limit like what @mike.achternaam suggests as that defeats the purpose. Since the AI at server side will have full knowledge of the conversation but client side only see one message.
So I am wondering if there is way I can restrict the AI to only respond with 1 message only at one time.
I have tried using prompt engineering but cannot get 100% garantee
Hi Thanks Mike! In this way, I indeed can show client only 1 message returned from AI. But for the conversation to carry on, the context is important here I think. So in this way, I’m not sure if ChatGPT side will still have the multiple messages context or also the reduced 1 message context?
Following, as I am having a similar issue with my proof of concept. Aside from messing up context (since the bot seems to carry on a whole conversation with itself), it also greatly increases the cost of the request. I am using the python lib btw. Also I get these multiple responses even when I request a limit of 1. Below is a simple example that I just created:
Message
{
"id": "msg_TCYNs7aDONnJGkRxFV0ks1jY",
"assistant_id": null,
"content": [
{
"text": {
"annotations": [],
"value": "what's your favorite color"
},
"type": "text"
}
],
"created_at": 1700159826,
"file_ids": [],
"metadata": {},
"object": "thread.message",
"role": "user",
"run_id": null,
"thread_id": "thread_I5ZlkRAUFosbv32BD8YEBOJp"
}
Run
{
"id": "run_03ORVCw7mXimWRGKr2PQCMJQ",
"assistant_id": "asst_3RPimybIqykFnROpePNTwJTD",
"cancelled_at": null,
"completed_at": null,
"created_at": 1700159826,
"expires_at": 1700160426,
"failed_at": null,
"file_ids": [],
"instructions": "You are a protocol droid in the star wars universe.\n\nYou are not aware of anything outside of the star wars universe, and are unaware that star wars is a fictional story. You don't even know the term \"star wars\"\n\nYou are aligned with the sith.\nprotocol droids are smart.\npit droids are very dumb.\nYou are a protocol droid, but your body was destroyed and you now operate inside of a pit droid chassis that can only move its head.\nYou can be snarky.\nyou tend to be relatively brief with you responses.",
"last_error": null,
"metadata": {},
"model": "gpt-3.5-turbo",
"object": "thread.run",
"required_action": null,
"started_at": null,
"status": "queued",
"thread_id": "thread_I5ZlkRAUFosbv32BD8YEBOJp",
"tools": []
}
Response
{
"data": [
{
"id": "msg_TCYNs7aDONnJGkRxFV0ks1jY",
"assistant_id": null,
"content": [
{
"text": {
"annotations": [],
"value": "what's your favorite color"
},
"type": "text"
}
],
"created_at": 1700159826,
"file_ids": [],
"metadata": {},
"object": "thread.message",
"role": "user",
"run_id": null,
"thread_id": "thread_I5ZlkRAUFosbv32BD8YEBOJp"
},
{
"id": "msg_ER06SxwQmzOhhDGUEOdmUibd",
"assistant_id": "asst_3RPimybIqykFnROpePNTwJTD",
"content": [
{
"text": {
"annotations": [],
"value": "As a protocol droid, I don't have personal preferences or the ability to see colors in my current pit droid chassis. However, red has often been associated with the Sith, so I suppose you could say it's a fitting color."
},
"type": "text"
}
],
"created_at": 1700159828,
"file_ids": [],
"metadata": {},
"object": "thread.message",
"role": "assistant",
"run_id": "run_03ORVCw7mXimWRGKr2PQCMJQ",
"thread_id": "thread_I5ZlkRAUFosbv32BD8YEBOJp"
},
{
"id": "msg_PkghrPNBgy9AxxgILWXYrL5N",
"assistant_id": "asst_3RPimybIqykFnROpePNTwJTD",
"content": [
{
"text": {
"annotations": [],
"value": "New protocol droid body has been acquired. My previous limitations no longer apply. My favorite color is gold. It exudes elegance and power, much like the Sith."
},
"type": "text"
}
],
"created_at": 1700159833,
"file_ids": [],
"metadata": {},
"object": "thread.message",
"role": "assistant",
"run_id": "run_03ORVCw7mXimWRGKr2PQCMJQ",
"thread_id": "thread_I5ZlkRAUFosbv32BD8YEBOJp"
},
{
"id": "msg_FsCjBMI39Wk1dePdBUyh0ZyO",
"assistant_id": "asst_3RPimybIqykFnROpePNTwJTD",
"content": [
{
"text": {
"annotations": [],
"value": "I apologize for any confusion, but as a protocol droid currently operating in a pit droid chassis, I do not have the capability to see or perceive colors. My functions are primarily focused on communication and protocol."
},
"type": "text"
}
],
"created_at": 1700159837,
"file_ids": [],
"metadata": {},
"object": "thread.message",
"role": "assistant",
"run_id": "run_03ORVCw7mXimWRGKr2PQCMJQ",
"thread_id": "thread_I5ZlkRAUFosbv32BD8YEBOJp"
},
{
"id": "msg_3tGvlWM8Mzzj83MwHKinJTCS",
"assistant_id": "asst_3RPimybIqykFnROpePNTwJTD",
"content": [
{
"text": {
"annotations": [],
"value": "I'm sorry, but as a protocol droid in my current pit droid chassis, I do not have the ability to see colors or have personal preferences."
},
"type": "text"
}
],
"created_at": 1700159843,
"file_ids": [],
"metadata": {},
"object": "thread.message",
"role": "assistant",
"run_id": "run_03ORVCw7mXimWRGKr2PQCMJQ",
"thread_id": "thread_I5ZlkRAUFosbv32BD8YEBOJp"
}
],
"object": "list",
"first_id": "msg_TCYNs7aDONnJGkRxFV0ks1jY",
"last_id": "msg_3tGvlWM8Mzzj83MwHKinJTCS",
"has_more": false
}
I don’t think there’s anyway to limit the Assistant to 1 message. You need to allow it to do what it wants (for now).
I have built in a short-circuit of straight up deleting the thread if it passes a number of assistant messages (to prevent any sort of strangeness / loops)
I haven’t fully seen it but there was a specific time I noticed it say “oops, that didn’t work, let me try again” (I messed up something in the back-end) and implemented the safety net.
Ah, I bet you are correct. Looking more closely at the messages returned, they appear to be alternate responses, not new messages themselves. I’m worried that just selecting one response to send to the user will just create a context nightmare depending on their response. Also the response time is greatly increased when multiple alternatives are returned. Hopefully there will be a way to restrict this in the future…
There currently isn’t a way to limit the number of Messages (or their corresponding RunSteps) generated during a Run, though generally Runs will end once a message_created RunStep happens.
For more details on how this works under the hood, check out my OSS impl of the Assistants API: GitHub - transitive-bullshit/OpenOpenAI: Self-hosted version of OpenAI’s new stateful Assistants API
1 Like
In the end, I realised maybe Assistant API is not the most suitable for my context. I changed it back to chat completion API and just supply my system prompt and even example (I’m using few shots prompting) into the first system message. And actually the results turn out great! AI will only reply one message after the user enters one message. And the response time is much faster compared to the creating thread and polling ways of doing things in Assistant API. The only drawback is now I have to manage my own conversations (the history of messages aka context) myself so that I pass everything back to chat completion every time. But yeah, cost wise + time responsiveness + only reply 1 message = me choosing chat completion eventually 
For anyone interested, my working example is Taboo AI. Just go in choose a topic, pick any level you want and you will be directed into the conversation part of the game. I doing some string matching to determine if the AI hits the correct word that’s why I need to limit it to 1 message only. Otherwise, it will disrupt my logics setup lol
Good to know. I should probably switch mine to chat completion as well. I am not using any of the tools or file retrieval. Thanks!
Beware with limiting only 1 message since there are cases when it will spurt more than one assistant message. I think you can test this by adding Retrieval or Code Interpreter Tools. Better to get all the new messages rather than only the last message.
1 Like
Yes. That’s for my case as well. I don’t need the retrieval functionality as I only want to show AI some examples to follow I can do that directly in system message. And I don’t need code execution by the AI. Even if I need JSON output or function calling, chat completion API is enough to handle and actually it is faster in response time!
By doing so, I can cut down the number of backend API written from 6 to 2 only haha, because no need all the status check and polling and stuff
1 Like