The Model Is Not the Bottleneck. The Routing Is.

I still route work by hand: three models, two agent sessions, and a text file of handoffs. A task arrives, I read it, decide whether it needs the fast model or the slow one, write the prompt, and pass it along. That decision is the most valuable thing I do all day, and it is the least systematized.

The obvious response is to wait for a bigger model. I don't think that is the bottleneck. The bottleneck is the routing decision: which kind of thinking a task deserves, and what evidence would make me change my mind.

Daniel Kahneman's fast and slow thinking is a useful metaphor here, but it is not a model-size chart. Fast thinking is not a small model. Slow thinking is not a large one. Fast is compiled: a trigger, a rule, a template, a cached playbook. Slow is deliberate: decomposition, multiple candidates, verification, a human checkpoint. A frontier model can be fast and wrong. A mid-tier model can be slow and right if the process around it is good.

Fast and slow are paths, not model sizes

The fast path should not call a model

The first surprise is that the fast path should usually not call a model at all. If a task has a known trigger and a known action, the fastest path is a function: when X, do Y, because Z. The model only enters when the trigger is ambiguous. That is why a cheap model with a big memory is still slower and less reliable than a good reflex. You are paying for inference to rediscover something you already knew.

Slow is a process

The second surprise is that slow is not "ask the biggest model." Slow is a sequence. Decompose the task. Retrieve the relevant context. Generate independent candidates. Verify them. Decide. A bigger model helps at every step, but it helps less than a tight harness. I have watched a well-scoped refactor go better on a mid-tier coding model with a checklist than on a frontier model with a vague prompt. That says more about the harness than about the frontier model.

Multiple models are useful, but not as a free-form debate. Independent candidates plus a verifier beat a round table. Debate tends to converge on the most confident voice, which is not the same as the best answer. Use debate for values and ambiguity. Use verification for facts and code.

Choosing a model is the easy part. The harder problem is deciding when the fast path has stopped being true.

A reflex is a bet

A reflex is a bet that the world repeats. Most of the time it does. The job is to notice the times it doesn't. Every reflex needs a life cycle, not just a definition:

trigger:
action:
scope:
confidence:
evidence:
last_validated:
failure_count:
owner:

A strong reflex can be promoted to a higher layer: personal, team, department, company. Promotion should require evidence across contexts, a low failure rate, and an owner who will answer for it. Repetition alone is not evidence. A reflex that fires a thousand times and is wrong ten percent of the time is not a strong reflex; it is a superstition with a good memory.

Weak reflexes need a way out. Some should decay. Some should be challenged. Some should be retired. The immune system metaphor is useful: white blood cells scan for contradictions, lymphocytes go after a specific bad reflex. Passive decay alone is not enough, because a stale reflex can sit in the fast path for months, quietly costing you the one case where it no longer applies.

The same applies to the slow path. A slow path that never becomes a reflex is just a permanent tax. The point of deliberate work is to compile some of it: a decision that was hard once becomes a checklist, then a rule, then a reflex. The system gets faster without getting dumber.

The human is the sensor

Then there is the part that is harder to engineer: the human.

The model has read more than I have. It has not sat in a customer call at 2 a.m., or watched a launch go sideways because one sentence on a pricing page was ambiguous. The gap is experience, not knowledge. The only way to give the system that experience is to record the decisions, not just the outcomes.

A preference is cheap. A decision trace is expensive, and worth it:

options:
evidence:
choice:
rationale:
outcome:
post_use_review:

I built an architecture assistant around this idea. It reads our system definition, our failure cases, and a set of positive and negative examples. It is good at producing options; judgment is still the human's job. What helped was not more documentation; it was the trace: what I saw, what I considered, why I chose, what happened, and what I would do differently. Positive and negative cases helped, but they were sparse. The trace was denser, and it was the only thing that let the assistant see the shape of a decision rather than the shape of an answer.

This is also why "human in the loop" is not a checkbox. A thumbs-up is an opinion. A two-week outcome is evidence. If you only collect preferences, you train the system to sound like you. If you collect decisions and outcomes, you train it to think like you in the situations you have actually survived.

A portfolio of local optima

The worry is local optima. It is a real worry. A system that learns from its own decisions can get very good at the cases it has seen, and very confident in the wrong places. The answer is a portfolio of local optima, plus a way to retire one when the world moves.

That means a few unglamorous things: a champion and a challenger, an exploration budget, a red team, a scheduled retirement review. Not a single global optimum. A set of local optima that can be recombined. Success, in a changing environment, is the best set of answers you can still revise, not the single best answer.

Four components

If I were building the routing layer today, I would start with four components:

1. A router. Score each task on novelty, ambiguity, stakes, reversibility, and dependency. Route to fast, slow, or escalate. The router should be allowed to say "I don't know."
2. A reflex registry. Every reflex has a trigger, an action, a scope, a confidence, evidence, a last-validated date, a failure count, and an owner. Promotion is a decision, not an accumulation.
3. A model portfolio. Model cards, not a leaderboard: what each model is good at, what it costs, how much context it needs, how it behaves with tools. Route by fit, not by rank.
4. A decision trace. Options, evidence, choice, rationale, outcome, post-use review. This is the data that teaches the system how you judge, not just what you say.

Start with one domain. Measure four numbers: reflex hit rate, escalation rate, cost per successful task, and rework rate. If the fast path never escalates, it is not fast; it is reckless. If the slow path never compiles a reflex, it is not learning; it is just expensive.

The schema is the easy part. The discipline is writing the decision down before you know the outcome.

The model is the CPU. The routing layer is the operating system. We have spent a lot of energy making the CPU faster. The OS is still a person with a clipboard.

That is the layer I want to build next.

Disclosure: I work on Wallaby Token, an inference platform for open-weight models. This essay is about the routing layer in my own workflow; it is not a product announcement.