Function response does not match tool specification

API
1

I specified a tool to determine the species of an object from a photo but the output contains a location? This is not part of my specification. Does it mean tool calling will randomly fail at producing the expected output format?

This was about 1 in 1000 failures.

The output:

tool_call  {'id': 'call_CyR62GiS01xfM96dNWp3tXD1', 'type': 'function', 'function': {'name': 'get_species_names', 'arguments': '{"location": "Bisbee"}'}}

Here is my specification for a tool:

{
        "type": "function",
        "function": {
            "name": "get_species_names",
            "description": "get the name of the species in the specimen from the label if it exists. In the case the label or image has multiple species return a string with the species in a comma separated list.",
            "parameters": {
                "type": "object",
                "properties": {
                    "species_names": {
                        "type": "string",
                        "description": "the names of the species",
                    }
                },
                "required": ["species_names"],
            },
        }
    },
2

Just ran some stats:

tool_responses = 156559
tool_output_errors = 26892

So this means 16% of tool responses are not conforming to the specification. Help?

These responses come from the batchAPI:

{
        "custom_id": f"{data['volume']}-{data['id']}", 
        "method": "POST", 
        "url": "/v1/chat/completions", 
        "body": {
                "model": "gpt-4o-mini", 
                "messages": message_payload(data['raw_label'], data['section_title']),
                "tools": tools,
                "tool_choice": "required",
                "max_tokens": 1000
            }
        }
3

what kind of object or photos are you sending? when the errors occur, what were the photos and what should be the correct responses? for example, the output with location, what is it supposed to be?

4

I’m not sending photos, only text.

The output should be “species_names” as is defined by the tool.

It’s hallucinating ‘location’ instead of following my spec.