BuiltOnJev

Jev vs LLM: a decision model is not a smaller text model

The most common question about Jev is whether it replaces your LLM. It does not — and framing it that way is why people end up disappointed. They solve different problems, and the interesting question is where the boundary sits.

Updated September 2026 · 7 min read

The actual difference

An LLM is a sequence generator. You give it text, it produces more text, one token at a time. Everything it knows how to do — answer, write, code, classify — is expressed as text coming out.

Jev is a scorer. You give it a state plus a set of typed questions, and it returns typed values: which option wins, what score on an ordered scale, what probability a yes/no is true. It never produces a sentence. There is no prose to parse and no output format to coax out of it.

  • LLM output is a string that you then interpret. If you want a label, you write a prompt, hope the model follows the format, and parse the result.
  • Jev output is a value with a declared type. The schema is the contract — you get an option key and a probability, not a paragraph containing one.
  • That difference is why one is measured in seconds and the other in milliseconds. Generating 200 tokens is work; scoring 200 options is arithmetic over them.

It is not either/or — it is a division of labour

The useful mental model: LLMs write, Jev decides. Any place your software has a branch it cannot express as a clean if is a candidate for Jev. Any place it has to produce language is not.

The jobReach forWhy
Draft a reply, summarize a thread, write codeLLMOutput is language. Jev cannot produce it at all.
Is this ticket urgent? Which queue? How risky is this diff?JevA typed decision. An LLM would answer in prose you have to parse.
Extract structured fields from free textJevTyped output by construction — no schema-violation retry loop.
Decide whether a long agent transcript is still worth keepingJevkeep / truncate / drop is a three-way choice, not a writing task.
A judgment so rare or so open-ended that defining the options is the hard partLLMJev needs the option set up front. If you cannot enumerate the answers, you cannot ask Jev.

Rule of thumb: if the answer is one of a set you can write down, it is a Jev job. If the answer is new text, it is not.

Our own pipeline is the worked example

This site reviews every submission with Jev. Paste a link, and one call decides two things: whether it is a Jev build at all (noul), and which of the 21 use cases it belongs to (choice).

  • One call, two questions. whether the link is a Jev build at all and which of the 21 use cases it belongs to are asked together and answered in parallel.
  • No parser, no retry loop. The category comes back as one of 21 known keys. There is no regex over model output and no “the model returned invalid JSON” branch, because invalid output is not representable.
  • What an LLM version would need: a prompt instructing the format, a parser, a validation step, a retry path for malformed output, and a decision about what to do when the model hedges instead of choosing.
The honest part
When the reviewer is unavailable (no API key, network failure), submissions fall back to a reachability heuristic and are labeled reviewed by fallback instead of reviewed by Jev. That is a real limitation of leaning on any external model — and the reason the badge on each listing says which reviewer actually ran.

These are the builds that made it through that pipeline, most upvoted first:

The cost math, and why it changes behaviour

The headline is that Jev is far cheaper per judgment, largely because output tokens are free and there are no output tokens to speak of. That matters less as a line item than as a design constraint removed.

ModelAccuracyCost / caseTime / case
Claude Opus 573.1%$0.176137.8 s
Jev67.8%$0.00040.07–0.5 s
Terra67.9%$0.030410.1 s

Reference answers in these evals came from two large models, not humans. Source: TypeSafe's published evals, read 2026-09-24.

When a judgment costs a fraction of a cent and returns in under half a second, you stop rationing it. You ask about every email instead of a sample. You check every tool call instead of the ones that look scary. You re-evaluate on every loop iteration instead of caching a decision made ten steps ago.

The counter-intuitive consequence
Jev being cheap does not make your bill smaller. It makes your system ask far more questions. The Jevons paradox is literally where the name comes from — make a resource cheaper and consumption rises rather than falls.

Where Jev loses to an LLM

On the same eval, Jev scored 67.8% against 73.1% for a frontier LLM. It is not more accurate. It is faster and cheaper and its output is type-safe — those are different axes from being right more often.

  • Accuracy is lower, and the eval is soft. Reference answers in these evals came from two large models, not humans.
  • You must enumerate the options. If the correct answer is not in your list, Jev cannot find it. An LLM can at least tell you your list was wrong.
  • No chained reasoning inside one call. Questions are answered independently. Multi-step dependent logic has to live in your code.
  • Text in, values out. No images, no audio, and nothing to read afterwards.

The practical pattern is to route: let Jev take the high-volume easy calls, expose its confidence, and escalate the uncertain ones to a frontier model. That is a strictly better use of both than picking a side.

Common questions

Is Jev an LLM?
No — but the honest answer is that it comes from the same family. It is a transformer trained with a different objective (RLCD, calibrated decisions) and optimized for a different output shape. It never produces free-form prose.
Can I replace my LLM with Jev?
Only for the judgment calls. Anything that produces language — replies, summaries, code — is out of scope, because Jev does not generate text at all. Most teams end up running both.
Does Jev hallucinate?
Not in the usual sense. The output format is constrained, so it always returns a valid option, score, or probability — there is no way for it to invent a field or emit broken JSON. But picking a wrong option is still possible, which is what the returned confidence is for.
Is Jev faster than an LLM?
Yes — typically 70–500 ms per call per call, versus tens of seconds for a frontier model on the same judgment. The speed comes from scoring options instead of generating tokens.

BuiltOnJev is an independent community project, not affiliated with TypeSafe AI. Specs and eval numbers come from TypeSafe's published materials; directory figures come from this site's own submissions and are updated as builds arrive.