We’re building a small “brainstem” for LLM agents. The idea is simple: the agent measures its own uncertainty while it’s fetching context and reasoning, and then chooses one of four moves—hold, think more, answer, or act. If the signal says “confidence is low,” it doesn’t bluff. It slows down, grabs better evidence, or just abstains.
What it is (short & sweet):
-
A lightweight control layer that wraps any LLM.
-
It looks at the context the model is using and computes a few signals—how scattered it is, whether sources agree, and how stable the neighborhood looks.
-
Those signals drive a tiny gate that’s conservative by default: silence/think first, answer/act only when the support is solid.
-
Everything the agent decides is traceable (you can see why it paused, why it answered, what evidence it used).
Why this matters:
-
Most hallucinations happen when the model talks with thin or conflicting context.
-
By turning “uncertainty” into a first-class control signal, the agent avoids confident-sounding guesses and risky tool calls.
-
You also get cleaner debugging: less “why did it say that?” and more “here’s the trail.”
Core characteristics:
-
Works with hybrid retrieval (keywords + vectors), but checks consistency before trusting results.
-
Has a notion of geometric stability to avoid drifting into random neighborhoods.
-
Runs a budgeted “sleep” step to tidy up memory over time (keeps things fast and auditable).
-
Ships with an append-only ledger so you can replay decisions later (useful for audits and post-mortems).
Where it’s useful:
-
RAG assistants that need to stay faithful to their sources.
-
Agents with tools (ops, tickets, automations) where a wrong click costs real money.
-
Regulated or high-stakes setups that prefer “no answer” over a polished error.
Why it could unblock some stubborn AI issues:
-
Puts a stop to “answer-at-all-costs” behavior by giving the model a real abstain path.
-
Treats uncertainty as behavioral logic, not just a number you print in logs.
-
Makes long-running agents less flaky by keeping memory coherent and decisions replayable.
Who else is thinking along these lines (roughly):
-
Uncertainty-aware LLMs (entropy/logit confidence, self-consistency voting, verifier/critic passes).
-
Selective answering / “I don’t know” policies and deferral.
-
RAG quality checks that require source agreement before responding.
-
Agent controllers that gate tool use when confidence is low.
If you’re into agents that choose not to answer when they shouldn’t, this is exactly that. Happy to compare notes with folks building uncertainty-aware routing, safer tool use, or audit-friendly agent stacks.