I wrote this post to share our journey building an OpenAI-powered form builder. I share what went wrong with our first launch, what we learned, and what we’re doing differently this time.
Hopefully it’s helpful to anyone else making an OpenAI integration for a SaaS product.
Our journey started in late 2022, when we excitedly added AI to our product. It flopped. Fast-forward 7 months and we just released our second attempt, with an entirely different approach.
OUR (FAILED) FIRST ATTEMPT
First, some context: our product is Fillout, a powerful form builder (think Google Forms, but with more styling options, better integrations, and advanced features). We first introduced AI into our onboarding flow to help solve the “blank canvas” problem.
The UX was simple: describe the form you want to make, with as much or as little detail as you want. Then, “AI” designs a form that you can add to and modify. We thought it would be the equivalent of a “form template”, but personalized to your exact use case.
Back in October 2022, using generative AI in-product was a fairly new idea. We launched on Product Hunt, pitching ourselves as the “AI-powered form builder” and got several-hundred signups.
Unfortunately, almost all the people who tried the AI onboarding immediately churned. The few who did stick around usually removed the AI-generated questions from their form pretty fast. As the final blow, a few of our existing users asked us to re-enable our old onboarding flow. That’s all we had see to be sure - we removed the AI feature completely, just one week after launch.
WHAT WE LEARNED
After the launch, we took a step back and watched a few of our friends use Fillout for the first time. Here’s what we learned:
- The experience was overwhelming. If you start from a blank form, you can gradually learn how to use the form editor as you make your first form. But with the AI onboarding, new users had to get acquainted with the Fillout editor and parse all the questions that were added to their form, at the same time.
- There was no ability to undo and iterate. Once you landed in the pre-built form, that was it. You couldn’t re-prompt Fillout to modify the form or undo some of the changes if they weren’t what you wanted. We understood this better a few weeks later when OpenAI launched ChatGPT. One of the things that makes ChatGPT great is the ability to iterate on a prompt if you don’t get the outcome you need - something Fillout lacked.
- The AI was a one trick pony. Even if the AI produced a helpful initial form, the user was left hanging, expecting the rest of the form building experience to be super-charged in some way. This was doubly true since we marketed Fillout to be an “AI-powered form builder” - lesson learned.
- We didn’t teach users how to prompt effectively. Many people gave generic prompts like Order form , which, unsurprisingly, yielded generic results. The few who did succeed provided details like the tone of the form, specifics on the questions they wanted to ask, and how long the form should be. Although we had a basic placeholder and tooltip, that clearly wasn’t enough to guide users to success.
WHAT WE’RE DOING DIFFERENTLY
The last few months has been a whirlwind of startups incorporating AI into their products. Though a lot of these new AI features have similar utility to our first Fillout attempt, we’ve seen several approaches that have been well-received and provide clear value (Notion and Retool come to mind).
Though we haven’t sat completely on the sidelines (we launched an experimental tool called graphmaker dot ai and an AI quiz maker), we’ve mostly held off adding AI to the core form building experience — until now.
How the new AI-feature works and why it’s different
- First, the experience is entirely in-editor. To trigger the feature, you click on the sparkle icon
in the top left corner of the screen. You can use it whenever you need it, or not at all. Because we don’t generate an initial form with AI, you get the benefit of gradually discovering the form builder without being overwhelmed. Once you start using the AI, you’re better acclimated to understand what is changing. - We show tips on how to prompt effectively and let you choose whether you want to add one question or multiple questions at once. This gives you more control over what the AI actually does.
- The AI is context-aware. Similar to how Github Copilot takes into account your most recently viewed code files, Fillout takes clues from your current form to ensure that the new AI content fits in as you’d expect it to.
- We stream partial completions from OpenAI to reassure you that something is happening and that you’re not just stuck on a loading screen.
- Most importantly, we clearly show you what questions were AI generated and give you the option to undo with one click.
The verdict’s not out yet, but so far in our own personal testing this new approach has proven significantly more useful.
If you want to try for yourself, you can sign up for free at fillout dot com.
Thanks for reading!
I’m around if you have any questions!