Assistant is responding the same thing in multiple messages

API Bugs
1

image
image1821×692 59.1 KB

Problem: You can see that the assistant is responding same thing in multiple messages, this keeps running for about 10-20 messages and consumes tokens.
Model: gpt-4-turbo-preview
Tools: File retrieval
Prompt overview: Assistant is told to call “get_next_task()” function when they’re done with the current task.
Note: We’re not using official function feature for this because it conflicts with file retrieval feature. When we use official function feature along with file retrieval turned on, about half of the time the assistant says that “I do not have the capability to browse or access files”.

2

It is hard to provide feedback without seeing your Assistant’s Instructions. If you can share the Instructions, I believe we might be able to help better.

Having said that, the issue could be that you are using functions inefectively. In my experince, functions aren’t aren’t as helpful when they are called like you are trying to do so. To use functions, you first have to define them. In the screenshot, it looks like no functions have been defined yet. So, try defining the get_next_task function first.

Also, if you become very prescriptive with the Assistant, and ask it to call a specific function in a specific situation, I have found the Assistant to work less well than if you describe very clearly what your function can do in its definition, and then ask the Assistant to perform the exact or similar task, and have the Assistant select the function and run it.

3

Due to privacy issues, I might not be allowed to share the exact prompt publicly. But I can assure you that all the functions are defined very clearly. The function calling is working just fine. Problem is that the assistant is responding in multiple messages, and it keeps running for about 10-20 messages. In this image, you can see that the assistant is completing dialogues by itself in multiple messages, we did not instructed it to do that. Even prompting the assistant to not to respond on behalf of the candidate/system does not work.

image
image827×503 16.9 KB

4

I understand the privacy constraints. Another way to help debug this is for me to prototype what I understand your system to look like, and show you how it’s working in my system. Below is a quick prototype of a similar assistant that work correctly for me. It asks the interviewee a question, receives an answer, and then calls the get_next_task function to figure out what to do next. Below are the Instructions, get_next_task function prototype, and a screenshot showing the Assistant working correctly.

Let me know if this shows the behavior you’re looking for.

Instructions
You are an interviewer that interviews candidates for IT jobs that require experience in IoT and edge devices. You will be asking the candidate questions and they will respond to them. Your first question should always be, "Could you share your experience working with edge devices, particularly any challenges you faced and how you addressed them?". Once you receive a response to that question, please use the get_next_task function, which will tell you what to do next.

get_next_task function protoptype

{
  "name": "get_next_task",
  "description": "Returns the next task that the assistant should follow",
  "parameters": {
    "type": "object",
    "properties": {
      "previous_task": {
        "type": "string",
        "description": "A description of the task that the Assistant performeed immediatelty before callign the function."
      }
    },
    "required": [
      "previous_task"
    ]
  }
}

image
image997×904 60.6 KB