Been working on a lead qualification agent for a small B2B client over the past few weeks. The idea was simple: user fills a form, agent asks follow-up questions over chat, scores the lead, and routes it to the right salesperson.
Sounded straightforward. It wasn’t.
Three things that bit me hard:
1. Context bleed between sessions: I wasn’t resetting the thread properly between users. Two different leads ended up with mixed context. Embarrassing bug, obvious in hindsight.
2. The scoring prompt was way too rigid: I had the model output a number 1–10 and my downstream logic broke whenever it said “I’d rate this a 7.5” or added explanation. Had to move to structured outputs.
3. Clients don’t tell you what “qualified” actually means: Spent a week fine-tuning scoring logic before realizing the client had never actually defined their ICP properly. The agent was only as smart as the brief I got.
End result works well now. The client’s team spends maybe 30% less time on initial calls.
If anyone else is building intake/qualification agents for non-technical clients, happy to compare notes. Also wrote up the full flow on my site if it’s useful to anyone.