Hello all - I’m prototyping an agent work flow in agent builder UI, have a hosted ChatKit instance running on my app. The workflow is working great until it gets to the agent node that calls the ClientTool function to render content on the UI. The function is invoked correctly by the agent, and content renders on the UI, but then my workflow doesn’t proceed to the next node. It seems to be stuck in a loop on the render node. When i try in playground, I get the same result - the workflow gets to the rendering node, calls the function to render on the front end, but then is expecting a response - when I send a success response, it just re-runs the same render node; stuck in loop.
The agent config is as follows -
#######
Instructions:
Your job is to compile content, call a client tool call and render the compiled content on the UI. After receiving confirmation of successful tool call, always:
Acknowledge that you’ve created the module
Offer to create additional modules
After calling render_to_canvas and receiving confirmation, always:
Acknowledge that you’ve created the module
Offer to create additional modules
The client tool to call is: [render_to_canvas]
User message:
Call the client tool [render_to_canvas] to render. content from {{workflow.input_as_text}}
#######
It seems from docs that there are lower level parameters that can be configured in the SDK. Since I’m prototyping I’m using the hosted instance for speed. Figured this would be basic functionality that should just work right of the box. I feel like I’m doing something wrong here - anyone come across this issue? Or if you’ve done function calls in the past on agents, is there an expected response structure that I need to follow?
I had history on - but anyway I created another end agent node that renders using client tools - chat history on. Also on the client side, returning just a plain {} seems to also help since my agent isn’t looking for data to be returned.
Also experiencing onClientTool issue. My onClientTool handler executes successfully and returns a valid JSON object, but the agent immediately breaks after that.
The client tool runs correctly, but ChatKit can’t send the result back to the agent.
PS.: I am using @openai/chatkit-react with Agent Builder client tools.
My client tools function call is rendering on the UI, but it is not very reliable. Often it fails silently which is frustrating because if there was any server side update we could invoke a retry action. Is there some best practices guide on how to work with client tools for UI rendering? I want to make sure I’m structuring the output and response correctly. At this point the reliability is so low that I’m considering not using client tools at all and instead persisting in db and fetching to render.
it is already few hours