HRH Projects: Intent Router

Community
, ,
1

A Simple Example-Trained Intent Router Using OpenAI gpt-oss Models

Recently, I made a simple example-trained Intent Router through vibe coding with ChatGPT.

In my current setup, although we have other models in store, the router mainly works well with OpenAI’s gpt-oss:20b and gpt-oss:120b models. The 20B model provides a faster option, while the 120B model is preferred for more demanding coding and reasoning tasks.

The router sits in front of these models and attempts to decide which one, or which other specialised endpoint, is best suited to each request.

Here is a short showcase of the current version:

The routing process

The main workflow takes place inside the Route Console.

First, the user enters a request. This can include ordinary text, code, or supported file types.

When Understand Intent is pressed, the request is not immediately sent to gpt-oss:20b, gpt-oss:120b, or another model.

Instead, the router:

  1. Converts the request into an embedding.

  2. Compares it with previously taught examples.

  3. Finds the closest intent examples using cosine similarity.

  4. Uses those examples to produce a structured description of the request.

  5. Determines the input, output, and capabilities required.

  6. Compares those requirements with the models in its registry.

  7. Recommends the most suitable compatible model.

The routing result includes:

  • Request type

  • Requested action

  • Information being used

  • Expected result

  • Required capabilities

  • Intent confidence

  • Closest learned example

  • Recommended model

For example, a PHP debugging request might be interpreted as:

  • Request type: code analysis

  • Action: debugging

  • Input: text and code

  • Output: corrected code

  • Required capabilities: coding and debugging

For this kind of request, the router may prefer gpt-oss:120b because it is registered as the stronger reasoning and coding option.

For a simpler or more time-sensitive request, gpt-oss:20b may receive a higher ranking because it can provide a faster response while still supporting the required capabilities.

The exact choice depends on the detected intent and the metadata stored for each model.

Understanding before execution

The Route Console separates the process into two actions.

Understand Intent analyses the request and displays the proposed route.

Run Selected Model sends the original request to the chosen endpoint.

I separated these stages so that the routing decision can be inspected before execution.

This makes it possible to see:

  • What the router thinks the request means

  • Which abilities it believes are required

  • Which model it recommends

  • Which examples influenced the decision

  • Whether its confidence is high enough to continue

When confidence is too low, or two possible meanings are too close, the router can pause instead of automatically selecting a model.

This is useful because even a powerful model such as gpt-oss:120b cannot fix a request that has already been routed using the wrong assumptions.

Teach Intent

The Teach Intent tab is where the router learns what different user requests mean.

Each training record begins with an example request, such as:

  • “Debug this PHP code.”

  • “Generate an image of a lighthouse.”

  • “Describe the attached photograph.”

  • “Transcribe this audio recording.”

The example is then connected to structured intent information:

  • Task family

  • Action

  • Input modalities

  • Output modalities

  • Required capabilities

  • Route shape

  • Reasoning complexity

The router therefore learns from complete examples rather than only checking for individual keywords.

When a new request is submitted, its embedding is compared with the stored examples. The nearest examples vote on the likely meaning of the request.

If the router misunderstands something, I can add or adjust examples in Teach Intent and rebuild the index.

The aim is to improve the router through understandable training records rather than hiding every decision inside a large prompt.

Models

The Models tab contains the endpoints available to the router.

In my current registry, gpt-oss:20b and gpt-oss:120b act as the main general-purpose models.

Each registered model includes metadata such as:

  • Display name

  • Model endpoint

  • Local or remote provider

  • Supported inputs

  • Supported outputs

  • Capabilities

  • Context length

  • Quality score

  • Speed score

  • Priority

  • Availability

  • Endpoint health

  • Measured response time

For example, I can register gpt-oss:120b with a higher quality and reasoning score, while gpt-oss:20b can have a stronger speed score.

The router compares this model information with the structured intent produced from the request.

A model must first support the required input and output types. Its capabilities are then compared with the abilities required by the task.

The remaining models are ranked using factors including:

  • Capability compatibility

  • Input and output compatibility

  • Quality

  • Speed

  • Registry priority

  • Locality

  • Endpoint health

This allows gpt-oss:120b to remain the preferred model for complex general tasks without forcing every small request through the largest available model.

It also allows specialised models to outrank both gpt-oss models when a request requires something different, such as image generation, vision, transcription, or speech.

Manual model overrides

Automatic routing can remain enabled, but the user can also select a model manually.

When an override is used, the interface still shows:

  • The detected intent

  • The automatically recommended model

  • The manually selected model

  • The final model response

This has been useful for comparing the router’s recommendation with direct tests of gpt-oss:20b and gpt-oss:120b.

It also makes failures easier to investigate. I can determine whether the problem came from the intent prediction, the model ranking, the endpoint, or the selected model’s answer.

How the router is built

The current router is a small hybrid experiment rather than a fully trained neural classifier.

It combines:

  • Example-based intent teaching

  • Embeddings

  • Cosine similarity

  • Weighted voting

  • Structured intent frames

  • Confidence thresholds

  • Capability matching

  • Deterministic model ranking

  • OpenAI gpt-oss model endpoints

The project itself was built largely through experimentation and collaboration with ChatGPT.

It has helped me better understand the layer between receiving a user request and deciding whether it should be handled by gpt-oss:20b, gpt-oss:120b, or a more specialised model.

It is not production-ready, but it has become a useful practical experiment in intent recognition and multi-model routing.

I would be interested in any thoughts on improving the training examples, confidence calculation, model ranking, evaluation system, or the balance between gpt-oss:20b and gpt-oss:120b.

2

V2: Routing Benchmark Lab

Following my first post about my example-trained Intent Router, I have now added a new Routing Benchmark Lab / Model Arena.

Youtube Showcase:

The original router focused on:

Request → Intent → Required capabilities → Model selection

V2 adds another question:

Which model has actually performed best on this type of task?

Model Arena

The Model Arena can run the same benchmark against multiple compatible models, including models such as:

  • OpenAI gpt-oss:20b
  • OpenAI gpt-oss:120b
  • Qwen3-Coder
  • Qwen3-Coder-Next
  • DeepSeek-R1

The models run sequentially so that latency measurements are not distorted by several large models competing for the same hardware.

Each benchmark stores the task, required capabilities, expected output, reasoning difficulty, and evaluation method.

Better Benchmark Marking

Instead of only checking whether an answer contains certain keywords, benchmarks can now combine several evaluation methods:

  • Required or forbidden content
  • JSON / structured output checks
  • Exact or numeric answers
  • JavaScript syntax checking
  • Output constraints
  • Blind manual marking
  • Blind model-based marking

Each evaluation method has its own weight, and the program combines them into a final score.

Blind Model Judge

One of the main additions is an optional AI judge.

I can select any suitable reasoning/chat model as the judge. In my current setup I use gpt-oss:20b.

The judge does not know which model produced an answer.

Instead of seeing:

gpt-oss:120b produced this answer

it sees something like:

Candidate-K7P4

The judge receives the original task, the evaluation criteria, and the anonymous answer.

It then returns scores for:

  • Overall quality
  • Correctness
  • Task completion
  • Instruction following
  • Relevance
  • Confidence

For example:

{
  "score": 94,
  "correctness": 97,
  "completion": 93,
  "instruction_following": 95,
  "relevance": 91,
  "confidence": 86
}

The 94% score becomes 0.94 and is combined with the other benchmark checks.

The AI judge is treated as one source of evidence, not ground truth. If deterministic tests strongly disagree with the judge, the Arena can flag an evaluation disagreement.

Repeated Trials and Measured Profiles

Benchmarks can also be repeated several times per model.

This lets the Arena measure:

  • Average quality
  • Worst result
  • Score variation
  • Success rate
  • Median latency
  • Number of samples

This creates measured model profiles instead of relying only on manually entered quality and speed values.

An Interesting Early Result

In one simple p5.js coding benchmark, several models achieved almost identical quality, but their response times were very different.

For example, a smaller coding model completed the task much faster than gpt-oss:120b while achieving nearly the same score.

This is exactly what I want the router to discover.

The goal is not simply:

Choose the biggest model for difficult-looking tasks.

It is:

Choose the model that gives the best measured result for this particular type of request.

Arena-Informed Routing

The Arena now compares:

Current registry ranking

with:

Arena-informed ranking

The normal router still considers manually configured quality, speed, priority, capabilities, and endpoint health.

The Arena-informed version can also consider:

  • Observed benchmark quality
  • Latency
  • Consistency
  • Number of samples

For now, this is only shown as a preview rather than automatically replacing the live routing decision.

Current Architecture

User request
      ↓
Example-trained intent recognition
      ↓
Structured RoutingFrame
      ↓
Compatible models
      ↓
Model Arena evidence
      ↓
Quality + latency + consistency
      ↓
Arena-informed model recommendation

The project is still experimental, but V2 moves the router from choosing models mostly from declared properties toward choosing them from measured outcomes.

A possible next step is to let accumulated benchmark results directly train the model-selection policy instead of relying mainly on fixed ranking weights.

3

SharingIntent

I’ve been working on my Intent Router
A while ago my dad worked on Agent GIF - Sharing Intent.

At some point the overlap became a little too obvious to ignore.

He made a format for packaging and sharing structured, machine-readable ideas and intent inside a GIF.

I made an Intent Router.

So naturally, I decided to one-up him and put the Intent Router inside an Agent GIF.

The result is an Agent GIF containing my Intent Router project/code as part of its structured payload.

My dad described it rather nicely as something like “a Git repo for ALSO non-code.”

So putting an Intent Router into a format designed around exchanging intent seemed almost compulsory.

His original Agent GIF - Sharing Intent thread explains the format and the thinking behind it much better than I can steal credit for:

Agent GIF - Sharing Intent:

To be clear, my Intent Router code - in its entirety - is inside the GIF above.
If you’re interested, you can just extract it from the GIF headers.
(ChatGPT could do that for you)

4

The inspect-before-run split is the strongest part of this design. In practice I would persist the proposed route—input digest, matched examples, confidence, registry version, and chosen endpoint—before execution. Otherwise a later registry change can route the same request differently, with no clean way to explain the earlier result. Low confidence should create a reviewable routing record, not just pause the UI.

5

Thanks, this is useful feedback. The Router currently contains quite a bit of what you’re suggesting already: the router separates inspection from execution, records the route decision, confidence, matched examples, selected model, alternatives, and also keeps low-confidence/abstained routes in the decision history.

The parts I don’t currently have are things like explicit registry versioning and an input digest tied to each routing record.

I can definitely see the reasoning behind those, especially for reproducibility and auditing, but I’m not yet sure how much practical value they would add for the current scope of the project. Could you elaborate a bit on where you think those additions become especially useful, or what problems you’d expect them to prevent in practice?

Thanks again for the ideas. They’ve given me a few things to think about.

6

The value shows up when the routing data changes after a decision has been made. Say a request went to model A yesterday, then the examples or thresholds are edited today. Without the input digest and registry version, the history shows the chosen route, but not whether a different result came from changed input, changed routing rules, or changed registry state.

I wouldn’t build a heavy versioning system for this. A hash of the canonical routing input plus a registry revision or commit SHA on the decision record is usually enough to debug the surprising cases. If the routes are short-lived and you never compare decisions across releases, then it may not be worth adding yet.