Assistants API not giving responses

API
1

So I’m seeing an issue where the assistant just doesn’t respond. I’m using #autogen here and this is the traceback:

Traceback (most recent call last):
  File "E:\Documents\repos\nbaGPT\src\main.py", line 93, in <module>
    user_proxy.initiate_chat(
  File "E:\Documents\repos\nbaGPT\venv\lib\site-packages\autogen\agentchat\conversable_agent.py", line 672, in initiate_chat
    self.send(self.generate_init_message(**context), recipient, silent=silent)
  File "E:\Documents\repos\nbaGPT\venv\lib\site-packages\autogen\agentchat\conversable_agent.py", line 420, in send
    recipient.receive(message, self, request_reply, silent)
  File "E:\Documents\repos\nbaGPT\venv\lib\site-packages\autogen\agentchat\conversable_agent.py", line 578, in receive
    reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender)
  File "E:\Documents\repos\nbaGPT\venv\lib\site-packages\autogen\agentchat\conversable_agent.py", line 1241, in generate_reply
    final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
  File "E:\Documents\repos\nbaGPT\venv\lib\site-packages\autogen\agentchat\groupchat.py", line 375, in run_chat
    reply = speaker.generate_reply(sender=self)
  File "E:\Documents\repos\nbaGPT\venv\lib\site-packages\autogen\agentchat\conversable_agent.py", line 1241, in generate_reply
    final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
  File "E:\Documents\repos\nbaGPT\venv\lib\site-packages\autogen\agentchat\contrib\gpt_assistant_agent.py", line 167, in _invoke_assistant
    assert len(run_response_messages) > 0, "No response from the assistant."
AssertionError: No response from the assistant.

At first I thought it was a timeout issue and autogen was just moving forward before the assistant had completed it’s run but that’s not the case. I have 3 assistants with autogen and 2 of them are working but one (nba_analyst) isn’t so I went in the playground and just gave it a message and the same thing happened in the playground. The run is indeed completing in “completed” status but no new message is added to the thread:

The returned run object:

{
  "object": "list",
  "data": [
    {
      "id": "run_SNlbaV8CZrqBpGe4mT474nx8",
      "object": "thread.run",
      "created_at": 1706040009,
      "assistant_id": "asst_FrIJVmoyyY1SPMxe5bpoEi3V",
      "thread_id": "thread_BJDUoCRPAYZ6zndIHDqE3cyR",
      "status": "completed",
      "started_at": 1706040009,
      "expires_at": null,
      "cancelled_at": null,
      "failed_at": null,
      "completed_at": 1706040059,
      "last_error": null,
      "model": "gpt-4-1106-preview",
      "instructions": <removed for brevity and security>,
      "tools": [],
      "file_ids": [],
      "metadata": {},
      "usage": {
        "prompt_tokens": 19335,
        "completion_tokens": 180,
        "total_tokens": 19515
      }
    }
  ],
  "first_id": "run_SNlbaV8CZrqBpGe4mT474nx8",
  "last_id": "run_SNlbaV8CZrqBpGe4mT474nx8",
  "has_more": false
}

List of the thread messages AFTER the completed run (nothing has been removed by me):

{
  "object": "list",
  "data": [
    {
      "id": "msg_mnDPoDE2lRUdjifLROCPbgz2",
      "object": "thread.message",
      "created_at": 1706040009,
      "thread_id": "thread_BJDUoCRPAYZ6zndIHDqE3cyR",
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": {
            "value": "What's the best GSW lineup that includes Kuminga?",
            "annotations": []
          }
        }
      ],
      "file_ids": [],
      "assistant_id": null,
      "run_id": null,
      "metadata": {}
    }
  ],
  "first_id": "msg_mnDPoDE2lRUdjifLROCPbgz2",
  "last_id": "msg_mnDPoDE2lRUdjifLROCPbgz2",
  "has_more": false
}

Notice that there is only the original user message and no message from the assistant. Also notice in the run that completion_tokens is at 180 indicating that there was a response but it was never added to the thread’s messages.

2

The next step to pursue would be to investigate the run steps.

If you have a complicated use of tools plus the internal tools, it may be that there was internal generation that triggered the API into backend “completed”, but the AI emitted tool recipient text that wasn’t passed.

The AI also could have decided that “endoftext” was the logical thing to output and thus no response to you.

1 Like
3

This assistant doesn’t have any tools. Also even if it decided “endoftext” was the output, shouldn’t I still get a new message entry to the thread (with just empty contents)?

I’ll go down the route of investigating the run steps. I’ve been using the Assistant API since it’s release and didn’t run into this issue until I started using autogen. The assistant is being built fine (hence me being able to test in the playground with it) so I’m not sure what autogen could be doing to cause this issue but I’ll also see if I can reproduce this issue w/o autogen just to be sure.

1 Like
4

Nevermind, you were right (and really good too lol). My DB got mixed up and I was indeed telling the assistant that it had functions in its instructions but wasn’t actually giving the assistant any tools. So it was trying to run tools that didn’t exist.

Just wanted to give a follow up for anyone coming along behind me who is just as bad as I am lol.

5

What do you mean telling it that it had functions? I’m trying to do something very simple, attach a file to a message and ask for some information about it. It will return a response, but only 10% of the time. Same code worked flawlessly a month ago. Something has changed…

6

For my issue I was telling the assistant, in it’s system message/instructions, that it should use foo function to do something. But I wasn’t actually passing any functions or tools to the assistant. So my error was painfully simple and I’m not sure if it applies to what you’re getting.

7

I’m telling it to use the attached file. It works 10% of the time, the rest of the time it either has a system_error or just returns completed with nothing. Was working 100% fine about a month ago, then slowly the error rates started creeping up, now it’s basically unusable and is eating tokens.