Using server_vad breaks tool use in realtime api

[tleyden]

Adding the following to my session.update:

    if turnDetectionMode == .semantic {
      session["turn_detection"] = [
        "type": "semantic_vad",
        "create_response": false,
        "eagerness": 0.6
      ]
    }

Seems to completely break tool use. It doesn’t matter if create_response is set to true or false.

Anyone else seeing this issue?

[sps]

Hi @tleyden,

I just tested this and was unable to reproduce it on my end.

Here’s the definition I used for the function tool:

This is the rest of the config and the conversation with the tool being invoked correctly.

@tleyden

"eagerness": 0.6

I noticed you’re using a floating value for the eagerness param, which can only have string values of low, mid, or high, other than the default auto.

[tleyden]

Thanks @sps , you found the issue! That eagerness param set to a number was completely breaking things. When I change it to ““eagerness” : “low”“, it works as expected.

I think the backend should be a bit more robust and validate the inputs, but this completely unblocks me. Really appreciate it!