How to get state information from a widget in AgentBuilder?

API
,
1

Hi,

I’m struggling to understand how an AgentBuilder Workflow can access the state/content of any widgets that it presents. A simple example is a widget with a few checkboxes, how does an agent (either the agent presenting the widget, or one later in the flow get the state of the checkboxes (assuming the user has updated them). Can anyone point me in the direction of any explanations / examples of an AgentBuilder workflow reacting to a user interaction with a widget?

Thanks!

David.

4 Likes
2

I’m also struggling with this. There is seemingly no documentation on how to use widgets with dynamic input fields (I.E a form ) with the GUI Agent Builder. My assumption is that its still to to-be-developed but hoping not the case. In my case, i have a widget with :

<Form onSubmitAction={{ type: "item.create" }}>

but i never see this item in the menu of available fields in later agents. I have tried declaring States to hold these values and it also does not work.

3

having the same problem too, I try to figure out how to interact with the widget

4

Does anyone have a response on how to get the data from the input? I’ve been struggling with this for days! :folded_hands:

1 Like
5

So here’s what I’ve come to. If you wrap a form in the component and you set a onSubmitAction like follows:

onSubmitAction={{

    handler: "client",

type: "selected_asset",

    payload: {},

}}

Then on your front end in your Chatkit (Chatkit React in my case) you can set widgets.onAction in your useChatkit configuration. The form events will go there with the information included in the payload property.

Where I’m stuck is there seems to be a race condition happening. The Widget is considered the output and at that point the agent hands off to the next step. But the whole point of a form input is to wait for the input before you move on to the next thing. I have this setup where I’ve given the agent a tool to ‘await widget action’ which basically is used from the front end to send back an action to the chat session to end the agent’s turn and then move to the next thing.

The documentation on this workflow is lacking. It seems like this should be a pretty integral part of agents…accepting widget input and using it to guide the rest of the conversation. Having widgets only be the terminal output of an agent doesn’t seem like it fits with this very common workflow.

There is a ‘stream_widget” utility mentioned in the ‘Custom implementation’ docs which says it allows agents to send widgets during their turn but I don’t see mention of that method in the ChatKit JS code.

It would be great to get more docs, a better set of JS examples and updates to the widget builder to handle for this case.

1 Like
6

Hey everyone, We discussed your request with our engineering team and at the moment we are working to build a mechanism to get state information about widget. We should soon launch it public as well. Hope that helps. Thank you!

1 Like
7

Is there any news on whether this has been released yet or when exactly we can expect it?