Most People on the Internet Miss What Jev is About
Cheap classification is the boring part. The interesting part is what a model that answers with types and confidence, instead of text, does to the harness around your LLM.https://github.com/gemanor/jev-code-review-benchmarkAlmost everyone who tried to explain Jev this week says the same thing. It’s…
Cheap classification is the boring part. The interesting part is what a model that answers with types and confidence, instead of text, does to the harness around your LLM.https://github.com/gemanor/jev-code-review-benchmarkAlmost everyone who tried to explain Jev this week says the same thing. It’s a model that does classification, very cheap and very fast.Some go one step further and say the big deal is structured output. If you’ve called any LLM API in the last two years, you know that’s not the big deal. JSON schema, tool calls, constrained decoding- every model API has it, every agent SDK has it. If that were the news, there would be no news.I did the same thing in my first reaction, so this isn’t me pointing at people. It’s the easy reading. The demo shows a classification question; the pricing page shows $0.042 per million tokens; you connect the dots, and you get “cheap classifier”.But it flattens what this model does. And if cheap classification was the point, it wouldn’t be much of a point. A small classifier you fine-tune on your own labels will be cheaper than Jev and probably more accurate on that one task. Nobody should be excited about that.Since the announcement, a couple of things have been running through my head that are much more interesting. They all come from the same place: what happens when you put a model like this next to the models we already use. That’s the System One / System Two thing. I want to show three use cases where it stops being a nice metaphor and becomes an actual architecture.First, what it isI think the key to understanding what Jev is about is understanding Kahneman’s System 1/2 and how modern harness engineering just became exactly this type of thinking. Kahneman’s System 1 is the fast, gut judgment. You look at a support ticket, and you know it’s a billing issue before you finish reading it. System 2 is the slow one, sitting and reasoning step by step.https://x.com/maxedapps/status/2100836066877952065?s=20TypeSafe took the name and made a model class out of it. A System One model doesn’t write text. You give it a “state”, which is basically any text or JSON you want, and a list of questions where the possible answers are fixed in advance. It answers all of them at once, and every answer comes back with probabilities and a confidence number.Three kinds of questions. Pick one option from a list (up to 255 options). Put the state on a scale you define. Or a yes/no that comes back as a number between 0 and 1.The interesting part isn’t the JSON. It’s what “no text generation” gives you.Nothing to parse. A type error can’t happen, and I mean can’t, because the answers were defined before the call. And the probabilities are calibrated, which is a word I want to make less scary: when the model says 0.8, then across many such answers it’s right about 80% of the time. They train it specifically for that. They call it RLCD, against RLHF for chat models and RLVR for reasoning models.A chat model, when you ask how sure it is, is overconfident by construction. It was trained to write answers people like, and people like confidence.Why does this matter so much? Because “the model is right 95% of the time” is useless for automation if it can’t tell you when it’s in the other 5%. With a calibrated confidence, you can write if confidence < 0.6: send to human and it means something.Speed and price come from the same decision. An LLM generates one token after another, so a long answer takes seconds and output tokens cost around 5x the input. Jev returns all the answers in one pass. 70 to 500 milliseconds per call. Output tokens are free. Input is $42 per billion tokens.Now back to the classifier argument, because it’s a fair one. If you have a narrow problem and you have labels — spam, sentiment, product category — train a classical model. It’s cheaper, and it’s good. Jev is not for that.Jev is for the question nobody trained a classifier for. You write the criteria in English, you get a real semantic judgment, roughly what a mid-size LLM would give you, and when the criteria changes you change a string, not a training set. Plus the confidence. That combination is what the next three sections stand on.Photo by Monica Sauro on UnsplashCaveats. It’s early access. Text only, mostly English. State is limited to 32k tokens. It reads very literally; it can’t count, do math, or compare dates, and their own docs say adversarial text inside the state can move the answer.On their evals, they compare it to GPT-5.6 Terra, not to the top models. They also say they can’t prove yet that the price isn’t subsidized. Keep all this in the back of your head.Use case 1: verify everythingEvery modern model can produce output you can verify. That part is not new. What we don’t have is verification at the same rate as generation, and the reason is simply cost.How do you verify an agent’s output today? Two options. Tests, type checkers, schemas, which cover exactly what you can write as code and nothing else. Or another LLM as a judge, which is a full generation call: seconds, expensive output tokens, and you get back a paragraph you now need to parse, from a model that says “yes, this is correct” in the same tone whether it knows or not.So in reality you verify a sample. Or only the final result. Or only the high-stakes things.With Jev, this flips. Take the artifact as state and ask 20 questions about it. Does this diff cover requirement 3? Does this quote appear in the source? Is there an instruction hiding in this retrieved text? Does this tool call match the plan?All 20 in parallel, a few hundred milliseconds, a fraction of a cent. Every answer comes with a probability, so the threshold is yours, per question.Now you can verify every step and not only the end. Every tool call, every retrieved chunk, every PR.This is where “faster” becomes the wrong word. 1000 verified answers in the time of one judge call isn’t a speed improvement; it means verification goes to places it couldn’t go before.TypeSafe’s own cookbooks are mostly this shape, by the way. Citation checking, jailbreak screening, scoring RAG passages before they reach the model, and a cascade where a cheap model extracts, Jev verifies, and the expensive reasoning model is called only when verification fails.The architecture changes with it. Instead of one big model that generates and self-checks inside a long chain of thought, you generate with whatever is good enough, fan the checks out to a System One model, and escalate only the low-confidence cases to System Two or to a human. That’s a different harness, and a much cheaper one.To be clear on the limits: Jev checks semantic things. It won’t count your tests, and it won’t compute a checksum. The deterministic checks stay in code. I think that’s fine; that’s the right split anyway.Use case 2: a “language” above the programming languageSecond thing. The bottleneck.Generation isn’t the constraint anymore. An agent can open 1000 PRs a day, no problem. The constraint is that nobody can merge 1000 PRs a day with confidence. Humans don’t scale to it. CI covers only what’s testable. LLM review at that volume costs more than the generation did and gives you back essays.So the traffic jam is at the merge gate. And it’s a confidence problem.The analogy I keep coming back to: we can all write software in JavaScript. But when we want something at scale that’s also predictable, we use a typed language with known output, and we accept a compile step for it. The compiler doesn’t make your program correct. It checks it against contracts you declared up front, fast, the same way every time, for every build and not for a sample.Now put the spec in natural language. Requirements, acceptance criteria, whatever you have. The agent writes the code. Then one System One call: spec plus diff as state, and typed questions. Does it implement requirements 1, 2, 3? Does it touch auth? Does it change a public interface? How big is the blast radius, on a scale you defined?High confidence on everything, it goes to the auto-merge lane. Low confidence on anything, it goes to a human, and the human sees exactly which question the model wasn’t sure about. That’s a type checker for intent, at compiler-like latency, at a price that lets you run it on every PR.And if this gate is cheap and consistent (Jev is very consistent, similar input gives you nearly identical numbers), then the spec becomes the thing you version. The natural-language spec lives in git. When a better model comes out, you regenerate the implementation, and the gate tells you if it still satisfies the spec.That’s what I mean by a higher-level language built on free text. The “compiler” is a generator plus a System One verifier, and the generated code starts to look more like build output than like source.Two caveats here. Jev generates nothing, so the “generates itself” part is 100% the LLM’s job; Jev is only the type checker. And because adversarial text can move it, you send the diff as state, not the agent’s description of the diff. Nobody has run this at 1000 PRs a day yet, me included. But for the first time, all the pieces exist.Use case 3: memoryMemory in agents today is mostly retrieval. Embed, similarity, top-k, put it in the context.It fails in two ways. It stuffs: the context grows, cost grows with it, and the model’s attention gets worse over the extra material. And it biases: a memory that’s similar but wrong for this task, or simply stale, pushes the model somewhere and nobody notices.The better memory layers add an LLM pass to rerank and filter. It helps a lot. But it’s System One work done with a System Two tool. Another generation call per turn, seconds of latency, and a judge that can’t say “I’m not sure”. As the store grows, the pass gets slower and less correct, so people cap it.Do it with a System One model instead. Cheap retrieval gives you 100 to 200 candidates. One call: state is the current task plus the candidates, and one yes/no question per candidate. Relevant to this task? Contradicts the current state? Same entity? Or one “pick the best” question over the candidate IDs to rank them.One round trip. A few hundred milliseconds. Every question is evaluated on its own, so adding candidates doesn’t hurt the other answers. You get a number per memory, and you decide the cutoff in code, or you only pass the confident ones forward.Their own reranking cookbook does exactly this over a keyword-search shortlist and takes top-1 accuracy from 5% to 18% on legal queries. The RAG one drops passages with injected instructions and flags the ones that contradict the question.So back to my three points. More correct, because selection is calibrated and consistent instead of “whatever the small model wrote”. Easier to pick what’s relevant, because relevance is now a number per item and the threshold is yours. And almost free compared to the generation call it protects, in money and in latency.Almost. It’s still a hop. And Jev has its own context rot; their docs say accuracy drops when the state is full of irrelevant stuff, plus the 32k cap. So no, you don’t dump the whole memory store in there. Retrieve first, judge second. It also doesn’t fix what’s stored; it fixes what gets picked, and it tells you when the picking itself is shaky. Which is the one thing memory layers today can’t tell you at all.This is where harnesses were going anywayLook at what a harness looks like after the last few months. A big reasoning model in the middle, RLVR-trained, chain of thought, long tasks. Around it, a ring of small fast models doing routing, classification, tool selection, guardrails, memory selection, judging.We got to System One / System Two by trial and error. Most of us built it ourselves because it raised accuracy and cut cost. What we didn’t have is a model built for the outer ring. We used small chat models and paid for text generation to get a label back.The outer ring is also where the volume is. It gets called 10x to 100x more than the reasoning model. Move that tier to something that costs $42 per billion tokens with free output, and a company whose harness is 80% System One, lead scoring, document review, ticket triage, moderation, gets a real extension of its runway.I’d hedge this one. The price is early, the rate limits change week to week, and TypeSafe themselves say they need time to prove it’s sustainable. But the direction is clear enough to plan around.It also finally gives the AI gateway something to do. Classify the shape of the request at the edge. System One-shaped goes to Jev, System Two-shaped goes to a reasoning model, unclear goes to a person. TypeSafe’s intent-routing pattern is exactly that. The gateway stops being a proxy with a cost dashboard.What I take from itJev was, for me, a good excuse to go back to basics on how these models are trained and how they sample. RLHF vs RLVR vs RLCD, one token at a time vs all at once, why a chat model is overconfident by design. Only after that do the use cases become obvious, and none of them was “classification but cheaper”.If you read Jev as a cheap classifier, you’ll build a cheaper version of what you already have. If you read it as a typed primitive with a confidence number that costs almost nothing to call, you’ll start moving things around in the harness. Verification and memory first, because those are the two places where the second call was always too expensive to make.This story is published under the Generative AI publication. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories. Let’s shape the future of AI together!Most People on the Internet Miss What Jev is About was originally published in Generative AI on Medium, where people are continuing the conversation by highlighting and responding to this story.Source: Generative AI Pub — Published — Category: Image AI