Hi All!
I am building with the assistant API. I instruct the assistant to do research work by generating a query based on the user input. It also has a function called research(), which will take the generated research query as input and return information.
In order to make the assistant more interactive, I specifically emphasized in the instruction that after the research query is generated, wait for the user’s response. If the user agrees, then call the function to gather information, otherwise iterate the process until the user statifies.
I created and tested such an assistant in Both the playground and actual API call.
In the playground, it behaved as I instructed, first generating a search query and waiting for the user’s feedback before calling the function.
However, when I do it with API call, the run.status direct goes from “in_progress” to “requires_action”. Even though it should first complete the run with generating a message, wait for the user feed back, then call the function.
Would really appreciate some help here.
Cheers
Hi and welcome to the Developer Forum!
If you get a return value of requires_action, what is the payload?
Thanks for the response, I am not sure what you mean with the payload.
Well, you get a response of requires_action, so what is in the message payload when that happens?
After the run.status goes to “requires_action”
I retrieved the message with
messages = client.beta.threads.messages.list(thread_id=thread.id).data
It only has one ThreadMessage, which is the original user message when I create the run.
I see.
RequiredAction(submit_tool_outputs=RequiredActionSubmitToolOutputs(tool_calls=[RequiredActionFunctionToolCall(id=‘call_3xWsRgA3opj2flypLKqKqj1v’, function=Function(arguments=‘{“query”:“generated query”}’, name=‘search’), type=‘function’)]), type=‘submit_tool_outputs’)
Ok, I may be getting confused here, can you post a log of the interaction please, showing what data is sent and what is received back, the Playground is simply a wrapper on top of the API, so if the playground is acting correctly then there must be some disconnect in the code flow.
Thank you for being patient with me. Have to admit I am really new here.
Would you like to see the log of the interaction of the playground? or during the API call?
If during the API call, could you please tell me how to get the log of the interaction?
thanks
Both would be useful, the standard way I try and debug is to put what I do in code then what I am sending in as data, then what I expect as output and then what I actually get as output, I do this for each stage and then I can build up a map of where an issue starts and potential fixes for that issue.
ok. I will start from the beginning.
-
create the assistant
assistant = client.beta.assistants.create(
name=“test”,
instructions=instruction,
model=“gpt-4-1106-preview”,
tools=[{
“type”:“function”,
“function”: {
“name”: “Search”,
“description”: “get information in xxx domain with a search query. The search query shall be in question format”,
“parameters”: {
“type”: “object”,
“properties”: {
“query”: {“type”: “string”, “description”: “a search query in question format”}
}
},
“required”: [“query”]
}
}]
)
-
Test in playground
I asked some specific question, the assistant generate a question for research and asked if I want to proceed. After I said yes, it called the function and ask for output from the function.
-
Test with API
assistant = client.beta.assistants.retrieve(assistant.id)
thread = client.beta.threads.create()
message = client.beta.threads.messages.create(
thread_id=thread.id,
role=“user”,
content=query
)
run = client.beta.threads.runs.create(
thread_id=thread.id,
assistant_id=assistant.id,
)
count = 0
while count <= 4:
run = client.beta.threads.runs.retrieve(
run_id=run.id,
thread_id=thread.id
)
print(run.status)
sleep(5)
count += 1
messages = client.beta.threads.messages.list(thread_id=thread.id).data
print(messages)
What i get is first in_progess run status, then requires_action as run.status
And there is only one user thread message in the messages.
Although what I expect is the run to complete and obtain a new assistant message in the messages.
Ahh ok, so it looks like the model is requesting the results from the search function, can you post the json object you get back?
Also who is providing the search function? is that something you have created?
Yes, I created the Search() function. Since the search function only takes one “query” parameter, the json I get back is Function(arguments=‘{“query”:“corrosion of aluminum-magnesium alloys in nitric acid”}’, name=‘Search’)
Which is in the right format.
It seems I have found out the cause.
run = client.beta.threads.runs.create(
thread_id=thread.id,
assistant_id=assistant.id,
#instructions=“Please use concise language.”
)
When I added in the additional instruction in run creation, even though it is a very simple instruction, the assistant could not behave accordingly. If I delete the instruction, then everything is fine.
I am very interested in such behavior. Does this show some vulnerability of the assistant’s instruction?
1 Like