How to get started with Agent Builder

I found doing the widgets and testing them out in chatkit through the starter app works.
see the flow here. https://youtu.be/VYuZcac2vaM?si=t9ARek5-_n—l_7

it looks nice but it’s a bit confusing.
i’ve created a simple chat app starting from the sample code: GitHub - openai/openai-chatkit-starter-app: Starter app to build with OpenAI ChatKit + Agent Builder and it works and it’s able to connect with the agent.

Now i’m stumped on a silly issue: i would like to pass the current date to the agent, otherwise the models thinks the current date is june 2024.

I have no idea how to do such a thing!

I might not be understanding well but creating a step where the workflow provides the user with different routes to follow, so the user chooses and the flow continues with that path is totally possible to do.

Create and agent that triages if this specific behavior is required, connect that to a if else, if that condition is true, connect with an agent node where you prompt something like “You provide the user with this three options etc etc “ now after the start node, the one at the beginning, you put another triage to evaluate the election of the user, if else to select the path and you put the path to continue from now on

1 Like

“Without coding” is definitely a promise you can’t keep

Suggestion: Add AI Assistant Sidebar to Agent Builder for Natural-Language Collaborative Workflow Creation

One of the main barriers for enterprises and developers when automating workflows is that Agent Builder still relies heavily on manually dragging nodes and configuring tools step-by-step, making standardization and debugging costly. I recommend implementing an AI-powered assistant sidebar in the user interface, enabling anyone—regardless of technical background—to describe business needs by voice or text (e.g., “create a multi-phase workflow for data validation, approval, and notifications”). The assistant would instantly parse instructions, generate the necessary nodes, connect APIs, design branching logic, and support interactive refinement. Such a feature would significantly lower adoption barriers and promote AI collaboration across industries—this is also a top request highlighted repeatedly in the developer community. I hope the team will prioritize this in the next product roadmap.

2 Likes

Has anyone discovered what the point of the end node is? ChatKit UI doesn’t do anything with structured output

Even the official template to compare files does not work if I upload attachments, I also tried to build one by my own, and it has below error whenever I attached a file (a text file) to proceed.

Workflow failed: Error running agent node: Connection error.. (code: api_error)

I have the same issue with mine:

Workflow failed: Error running agent node: Connection error.. (code: api_error)

Did you find the solution to this?

I also have the same issue. Even if I create a fresh Agent Workflow with a single node as something simple like “Describe this image” it fails. If I don’t attach an image, the agent will respond as expected, but if I do attach an image it fails with Workflow failed: Error running agent node: Connection error.. (code: api_error)

Edit: Just to clarify, images work fine in the SDK, this is entirely an issue with the preview within the Agent Builder interface. So if I make a python script to execute the workflow, it will handle the images, it’s just if I click “preview” and then the paperclip to attach an image, the preview run fails.

after one day it just works without any change, I believe something was updated on the backend. Though the agent builder seems to only accept PDF file at attachments with prompts if I am not mistaken, can someone confirm this?

I’m trying to have an agent handle nuanced user input mid workflow. Has anyone been able to do this beyond simple user approvals?

Hi! I’m pretty new to this so thanks for your patience– I’m getting started with building a medical advice agent, and uploading official medical guides as files so it gets its info from credible sources. When i asked my agent to tell me where the advice it’s giving me is coming from, it crashed and only spits back this error now– any ideas on what’s happening? thank you!

Error:

Workflow failed: Error code: 400 - {‘error’: {‘message’: “Item ‘msg_01a2a0c5dcc8f15100690d202da08081958141d0ca5f2a988f’ of type ‘message’ was provided without its required ‘reasoning’ item: ‘rs_01a2a0c5dcc8f15100690d201f1e708195974f8bfdc390abc1’.”, ‘type’: ‘invalid_request_error’, ‘param’: ‘input’, ‘code’: None}}. (code: user_error)