Jev in the agent loop: routing, guardrails, context

An agent's expensive calls are the interesting ones to talk about and the minority of what it does. Most of the work is tiny judgments: is this urgent, is this risky, is this still needed. Those are the calls a decision model is for.

Updated September 2026 · 7 min read

Why the loop needs a cheap judge

An agent loop makes a lot of small decisions. Which model should take this request. Is this shell command safe to run. Is this tool result still worth carrying. Each one is a real branch, and each one currently has only two implementations available:

A hand-written rule. Fast and free, until reality is messier than the rule. Then it is wrong quietly.
A frontier LLM call. Flexible and accurate, but you are paying seconds and real money to answer a question with three possible answers — and you have to parse prose to get the answer out.

A decision model is the third option: typed answers, milliseconds, fractions of a cent. That is what makes it viable to ask every iteration instead of sampling.

The three patterns

TypeSafe publishes three agent patterns. They are worth taking literally, because each one replaces a specific piece of brittle code:

Pattern	What it replaces
Route to the right model	Grade a request's complexity and send it to a fast, balanced, or frontier model.
Guardrail tool calls	Before an agent runs a shell command or edits a file, score the risk and allow, confirm, or block.
Compact context	Decide keep / truncate / drop for each stale tool result so long sessions shrink without a lossy rewrite.

Routing: grade the request, then pick the model

Model routing is the highest-leverage pattern and the easiest to get wrong. A router that sends everything to the frontier model costs what it always cost. A router built on regexes saves money until it meets a request that does not match, then sends it somewhere it does not belong.

Jev fits because the question is small and closed: given this request, is the complexity low, medium, or high? That is a choice with three options, and the per-option probabilities tell you when the call was close enough that you should just escalate.

The threshold is the design

Do not send the model's pick straight through. Send it through a confidence threshold: above it, route cheap; below it, route expensive. The probability is the part that makes routing safe rather than merely cheap.

Guardrails: score the risk before the tool runs

This is the pattern with the clearest payoff. Before an agent executes a shell command or writes a file, ask a noul: is this risky? You get a probability, and you turn it into a three-way gate — allow, confirm with a human, or block.

Fail closed, not open. If the guardrail call itself fails, the safe default is to confirm rather than allow.
Score, do not classify. A probability lets you move the line as you learn, without rewriting the guardrail.
Same idea for prompt injection. TypeSafe lists LLM guardrails as a first-class use case — scoring whether an input is attempting an injection is the same shape of question.

Context compaction: keep / truncate / drop

Long sessions drown in stale tool results. The usual fix is to summarize and lose detail. The decision-model version is to ask a three-way question per result — keep it verbatim, truncate it, or drop it — and never rewrite anything.

That is a choice with three options per item, which is exactly the kind of high-volume, low-stakes judgment that gets ruinously expensive when it is a frontier LLM call. It is nearly free here.

Agent builds in this directory

These are the listings Jev sorted into Agents & Browsers — real submissions, not examples we invented:

If you are building an agent and want to see the whole taxonomy, the full breakdown is on the use-cases page.

Common questions

Should every agent decision go through Jev?

No. Use it for small, high-volume, closed-set judgments — routing, risk gating, context triage. Anything that produces text, or where the answer set is not knowable in advance, belongs to an LLM.

How is this different from just prompting an LLM to output JSON?

The output type is the contract rather than a request. There is no format instruction to write, no parser, and no retry path for malformed output, because an invalid answer is not representable.

What is the most common agent pattern?

Model routing: grade a request's complexity, then send it to a fast, balanced or frontier model. It is the pattern with the clearest and most measurable cost impact.

Is Jev fast enough to sit inside a loop?

Typically 70-500ms per call, which is fast enough for per-iteration checks but not for anything you would call real-time control.

BuiltOnJev is an independent community project, not affiliated with TypeSafe AI. Specs and eval numbers come from TypeSafe's published materials; directory figures come from this site's own submissions and are updated as builds arrive.