Playground issues calling function

All,
i am having problems with submit button in playground.
sometimes my function is called twice, sometimes submit button is greyed out, sometimes it is here but not doing anything.
does it mean that something is wrong with my function or?
thank you
YK

I am assuming you are using Assistants API in the Playground.

sometimes my function is called twice

Yes, sometimes functions are called twice.

sometimes submit button is greyed out, sometimes it is here but not doing anything

You will need to input the mock output of your function to continue the run.

How to produce retired information? Is log data sufficient?
I cannot repro it on a stable basis

Thank you
Yk

You are given the tool response and possible suggestion for output.

Screen Shot 2024-01-16 at 9.06.48

{ “number”: 5 } // ← this is the response when the function is invoked

{ success: “true” } //<-- suggested output that you need to write then submit to continue the run

Thank you for your quick response!
Below is my situation, please let me know if my un.

Function definition includes input parameter(s), such as

parameters”: {

“type”: “object”,

“properties”: {

“location”: {

“type”: “string”,

“description”: “Street name or interception”

},

“unit”: {

“type”: “string”

}

},

While real implementation should submit information in the next manner:

{

“tool_outputs”: [

{

“tool_call_id”: “xxx”,

“output”: “Hollywood”

}

]

Playground makes next output suggestions see below is it playground hint?)

User

get camera near me

Run instructions

Please provide street/ave name of intersection

get_camera_by_location({“location”: “South St and Broad St, Philadelphia, PA”})

get_camera_by_location({“location”: “Walnut St and 11th St, Philadelphia, PA”})

get_camera_by_location({“location”: “Market St and 7th St, Philadelphia, PA”})

Submit‍

Three calls above!

/* typing in {“location”:”Hollywood”} */ in ALL THREE enables submit button, but nothing happens

What is in {output:”success”} purpose? Is it for playground only?

Thank you for your help

YK

Three calls above!

Yeah, it happens. If the location is not found in the messages or information you provided, this is hallucination.

What is in {output:”success”} purpose? Is it for playground only?

That part is the one submitted back to the API. It is just a suggestion. You can rewrite your output like this:

{ "status": "success", "location": "Hollywood"}

It is useful when say the user gave wrong location.

{ "status": "error", "location": "belmont street", "message": "unknown location or camera not found."}

I am not sure why you did not received any reply from the AI. Perhaps it depends on what your function is. Here I tested your sample function.

{
  "name": "get_camera_by_location",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "Street name or intersection."
      }
    },
    "required": [
      "location"
    ]
  },
  "description": "Get traffic surveillance camera based on given location."
}

Here are the sample convos:

  1. Success
    Screen Shot 2024-01-17 at 10.23.38

  2. Error
    Screen Shot 2024-01-17 at 10.24.54

1 Like

Thank you very much!
I will try and let you know.
Any chance for you to try in my environment?
Or debugging session?
Thank you
YK

1 Like

Same problem.
do you use the same tols and model?

Additional information:
looks like double function request is caused by two files.
i did upload file in FILES section below - i thought that it is specific for assistant?
however, the same file ended up in general file section.
as soon as i remove file from assistant, i got a single function prompt.
something is not right here.
Submit button does not do anything - i tried it from multiple machines

about Submit button - it does not take copy/paste. you need to take several trailing characters - at least in Chrome browser.
it is easily reproducible

YK

I tried to test it with file attached and Retrieval ON but I cannot recreate the problem. Although sometimes, in other Assistant, I also get it, however in this case, I don’t.

Perhaps it depends on the Instructions?

You are a helpful personal assistant.
You have the following tools you can invoke depending on user request.
- get_camera_by_location: when the user wants to search for traffic camera by given location.

Here’s some of my test results:

1 Like

Thank you for your help!
so i was able to navigate through some “features” :grinning: of Playground and i started to work on my use case - find red light camera on known location or near my current location

i added two functions (no file yet)
get_my_location
get_camera_by_location

The name of my assistant is Red light assistant :grinning:

Now i see responses below (my comments/questions are in italic):

User
get camera near me

Run instructions
Please provide street/ave name of intersection.
get_my_location(
{})
)
“Sheridan”

Looks like GPT does some verification based on response. it least, it does not take something like asdf :rofl:.
Does it do verification against real data? can i limit verification only to the file(s) that i upload? Or do some flexible validations?

get_camera_by_location({“location”:“Sheridan”}))
{“status”:”success”, “location":“Hollywood”, “lat”:80, “long”:60}
Red light assistant
The nearest traffic surveillance camera is located near the intersection of Hollywood Avenue and Sheridan Street.

Looks like tool tries to combine provided input value with response value. is there a way to tell system to ignore input value or to control “combining”?

Thank you very much for your help

YK

Looks like tool tries to combine provided input value with response value. is there a way to tell system to ignore input value or to control “combining”?

Perhaps not. Depending on its understanding of your previous query, it will either accept or reject your tool output. Sometimes, if it feels like the tool output is totally unrelated to the query, it will call the function again.

However, in real implementation, you probably won’t have to deal with it. Surely, you will get the location somewhere, perhaps based on the texts, or you might need to update your function to remove “location” if this value will be coming from internal API call.

And the question above?
In general, how much control do i have?
Thank you
YK

Does it do verification against real data? can i limit verification only to the file(s) that i upload? Or do some flexible validations?

You will need to do the verification yourself inside the function handler. For example, if you just submit “get camera near me” without any context, what is “near me”? if the AI has no reference for it, it will hallucinate some values. But for example, you attach an additional instruction that currently you are at point A, it should give you the correct location.

What is a purpose of the arrached file and retrieval? I think that i am missing something.
I do have function get_location that is being called to provide “near me” location.
However, i i provide asdf it does not take it - but if i provide Hollywood - GPT accepts it.

I think that is the question, will the API know how to use the file in the retrieval as additional instruction for validation? If I think about it, perhaps not, maybe unless you told it to. I read from another post asking the same question, if they can use the File as additional instruction. I want to say, no, but I need to verify it by testing. I have been reading some, specially those using custom GPT, that they somehow use the files like that.

Custom Api is a very powerful tool, however, in accordance to your previous answer it is a bit limited (combining input/output etc). But of course my custom Api can use files, databases etc without uploading. So i guess the question is: what is a purpose of files and retrieval tool? If the purpose is to limit search? To create search pririty? Anything else?

And how to use files properly? I read that files are indexed by vector db, is it correct? Just upload file seemingly does not have any affect. Besides, even if i upload a file on an assistant basis, looks like it is going to be shared across all assiatants

Thank you for your help
YK

When building a function call within the playground, where do you store the actual function that gets executed? I see where to store the description but don’t understand how it would actually call that functions content.

@glt32 ,
in accordance to my current understanding (of course, somebody from GPT team needs to confirm it) developer does not provide implementation of a function.
Rather, developer provides definiton of that function. When GPT thinks it needs more information, it tries to find a suitable function based on name, parameters etc. If such a function exists (from GPT point of view) it responds to a get run status call (https://api.openai.com/v1/threads/{{thread_id}}/runs/{{run_id}}) with "status": “requires_action” and additional information in “required_action” node of the response.
In this moment developer can call any custom API, db search etc to get relevant information. That relevant information is returned as Submit the function response (https://api.openai.com/v1/threads/{{thread_id}}/runs/{{run_id}}/submit_tool_outputs).
in other words, GPT knows that clarifying functionality exists and asks for it - it is developer responsibility to get necessary data and provides it to GPT.
In Playgroung you simply type in information that you should get by your own way.
Please let me know what you think.
I am also waiting for replies from @supershaneski

YK