Is it possible to have the user interact with the agent?

I’m playing around with the new agent builder.

Is there any way to make a user interact with parts of the flow? fx an agent needs to gather information from a user and once satisfied it needs to continue? I tried with a while loop and set the global state, but it never broke out of the loop.

2 Likes

did you look at the Agent Builder → Templates → Planning example?

you can see that the “while loop” is implicit in how the agent is wired to a condition. It does something mostly like what you said. Hope that helps!!

Hi @mcfinley, thanks for this. Would you happen to have any links to your suggested reading? I can’t find any examples or much on the while loop except from this https://platform.openai.com/docs/guides/node-reference#while - I tried to go throught the examples in the agent builder but none of them used the loop.

I basically have this very simple setup

I got three stories I want the user to select one before moving on.

I got a couple of global variables set up:

The three stories:

  • Story1

  • Story2

  • Story3

  • A boolean value to check if the user has chosen a story (set to false by default)

  • A string for the selected story.

Next, I have my while loop - I assume it’ll be running as long as my “storyhasbeenselected“ is false?

Then my story selector agent has this prompt:

You have these three stories the user needs to choose from:

# Option 1
 {{state.story1}}

# Option 2
 {{state.story2}}

# Option 3
 {{state.story3}} 

Get the user to choose one and output that story before you move on.
Out put as a boolean if the a story has been selected aswell as the story if it has been selected.

and an output set to JSON with the below schema.

The idea is for the agent to log whether the user has selected a story and if so which one.

Finally it sets the global state to true or false based on if the user have selected a story.

I’ll imagine if it’s false the agent will try again for user to select a story, if true, it’ll continue in the the flow.

However my agent never stops to take in a user value, it just goes in a continuous loop, prompting itself over and over again. So guess I’m trying to first and foremost to figure out, how to stop the agent to take in some input?

1 Like

did you take a look at GitHub - openai/openai-chatkit-advanced-samples: Starter app to build with OpenAI ChatKit SDK ?

Try coding up one of those in a super simple case that uses the template example with no changer. There’s a paradigm shift here so your confusion is understandable! I would start super simple and try one thing at a time, establishing a foundation of exactly how each building block works. I would even recommend just doing some API-based testing in a notebook as well.

1 Like

It’s probably simple to test with code, however, my coding skills are pretty much non existent, hence I really took to the visual builder. :pensive_face:

that’s actually amazing… you’re exactly who the tools are intended for - but you just need to work up slowly. Start with the most basic example - a simple agent (three blocks in the UI - start, agent, end). Preview it and get it to talk with you in natural language. See how multiple turns work, what it remembers, what the component takes as iput and what it puts out. Then add a condition - maybe tell the agent to output DONE when the user says something, and test for the output being DONE. Then ask the agent to make a structured output - something with more parts like whether the conversation is DONE, and whether the user is happy or something like that. Use a while loop to test for happy and make the agent keey truing until user is DONE and HAPPY. Work up slow, looking at what each component does - inputs, outputs, impact on the preview UI.

Its a little rough because the kit is so new but it works and you will learn 100 little things that will let you stand up the agent you envision. But go slow! you can’t tell what’s wrong when you try to do it all at once.

1 Like

I’ve been trying to build my chops. I think at a very basic level, I can’t find any way to let the user make an input except for user approval and the start text. It’s like dominoes that fall. Which is great for certain things, but if you need some more nuanced input from the user, it’s a bit limited to always just having to use hardcoded user approvals. I wonder if this is even possible?

yes! just make a variabe in teh structured output widget that contains what you want… the model will be very subtle in its level of understanding. Like you could have a variable “user_changed_theri_mind” and the model will correctly look for a change in sentiment or intent during the chat thread. with LLMs always try to get to the point where you can tell it what you want in natural language if you can. then get it to tell you back in a digital form like a variable so you can code against it.

1 Like

But how do I get the user to input “change of mind”?

  • I’ve tried to instruct the agent to stop and wait for input. It asks the question and continues.

  • Then I tried the while loop with an agent and a set state. The agent outputs a JSON object containing a single boolean value, either 1 or 0, depending on the user’s request. The state sets a global state to a 1 or 0, and the while loop should kick out of the loop once it’s set to 1, but the agent never stops to ask the question that will set the state to 1, so it just continues to loop.

So I guess any pointers on getting an agent to stop and take a user input would be absolutely amazing, as I’ve spent a couple of days trying to do this - and I’ve rust been running my head against the wall.

here’s a simple example. its a booking agent for a hair salon.

“start” happens every time a new user calls in (or opens a chat window, or sends an email)

the agent retains its state - there’s no need to loop anywhere because that’s under the hood.

the instructions for the agent are just “You’re an answering machine for a busy hair salon. Your goal is to make appointments.” but the magic is in the structured output for the “Agent” component:

The “if” block just tests to see whether the user is finished, and if they actually want to book.

When I run this in preview, I can have a nice chat with the agent about whatever I want, telling it details and changing my mind along the way. It will loop on its own, all I have to do is use the IF block to decide when its time to actually book an appointment.

Any clearer? you might be making it too hard. Let the language model inside the Agent block fill in a structured output for you, then check values on that output to determine actions.

3 Likes

also, check out this great post: OpenAI ChatKit: A FULL agentic chat experiences with One API Route and One Hook! | by Itsuki | Oct, 2025 | Level Up Coding

1 Like

The way you did it was how I was hoping it would work.

However, I did exactly as you did, but mine literally runs through the workflow without taking cues from the caller and ends as with no booking.

I wonder if it has anything to do with the model? I’m just using GPT-5, but also tried with Gpt-5-chat. Or is it because it’s not suppose to stop in the preview?

so you typed “hello” and it said there’s no booking, and the conversation is not over… that’s correct! Give it another variable “message_to_user” or something like that and it will give you the reply for chat.

you could say “I want to book kathy on thursday at 3” and it should indicate that the booking is done but the call is not over. then you say “bye” and it will indicate the cal is over… The loop is the user talking to the agent. ChatKit is holding on to the state so you can send it multiple messages. Hope that makes sense.

1 Like

It all makes sense what you are saying, and thank you for staying with me on this.

However no matter what I try to start of it runs through the whole workflow. No back and fourth chat.

I tried your suggest to ask kick off the conversation with more context.

But ran through the workflow without stopping.

I tried “I want to book kathy on thursday at 3”, but did the same.

Then I tried to add another variable as you suggest, but didn’t change much either.

Had a little look here seems to be more struggling with the same.

1 Like