TypeSafe's Jev CAN behave like a chat model, ish

When I went through TypeSafe's onboarding, the API dashboard asked me a quiz question about whether I could chat with Jev. I had to give the accepted answer to get access. I'm paraphrasing the question, but the experience was memorable for exactly the wrong reason.

TypeSafe presents Jev as a new class of model, with a new architecture and a training method called Reinforcement Learning for Calibrated Decisions, or RLCD. Its launch advertises 40-200x speedups and headline workflow results of 193.6x faster and 444.6x cheaper.

I didn't find the comparison convincing. I went into this expecting that ordinary language models, used properly, would undermine the price and speed story, and that a small local model would get reasonably close on a classification task. The results were broadly what I expected.

My impression of the marketing is that it is aimed at leadership who may not know enough about model inference to question the distinction being sold.

First, make it talk

Jev accepts state, instructions, and possible answers, then returns decisions and probabilities. Its Choice primitive allows up to 255 alternatives.

For my alternatives, I supplied tokens. The question was which token should come next in a conversation. I selected from Jev's returned distribution, appended the token, and asked again. Repeat until the end-of-sequence token.

That gives us an autoregressive decoder.

The 255-option limit is awkward when your tokenizer has roughly 30,000 entries, so I built candidate selection around it. A MiniLM WordPiece tokenizer supplied the vocabulary; corpus n-grams from WikiText-2 and word/POS transitions from English EWT supplied plausible continuations. Jev received the shortlist and made the final selection. There was no second neural language model writing the reply.

For larger shortlists, I split the candidates into windows, asked Jev to evaluate each window, and put the survivors into a final Choice. At 2,040 candidates, that means eight screening groups before the final selection. This machinery exists because of the API's limit. An unrestricted next-token distribution would make the process much simpler.

It produced meaningful text, including an answer to the capital-of-France question, and it also produced fragments and repetition. The prototype ran at roughly one selected token per second. Increasing the shortlist didn't reliably improve the output, because the screening process can discard useful candidates before the final choice ever sees them.

Jev's own documentation acknowledges that chaining choices can force text generation, and says it is slow and poor at it. Good. That makes the onboarding distinction even funnier.

I don't know Jev's internal architecture or the shape of its output head. I wouldn't be surprised if a broader vocabulary output were possible with a different head or interface. The observed behaviour is compatible with a model that has language capabilities but isn't trained or served for conversation.

The useful direction of this experiment, though, is the reverse. Making Jev generate text requires quite a bit of plumbing. Making an LLM return a class distribution is easy.

Stop making the model write its probabilities

TypeSafe's comparison uses an LLM wrapper that asks for structured decisions with probability values. The launch article acknowledges that this is slower and more expensive than requesting decisions alone. Its adapter also includes normalization for generated distributions that fail to sum to one.

So the LLM is generating labels, decimal numbers, punctuation, and JSON structure. Those numbers are textual estimates. They are not the probabilities in the model's output distribution. Then the cost of writing that table becomes part of the argument for a special class of decision model.

An LLM already computes scores over possible next tokens. If you want it to choose between 77 classes, assign each class a distinct token, describe the mapping in the prompt, and read those 77 scores. Apply softmax and you have your distribution. Serialization is ordinary application code.

Let the model's final hidden state be $h_{\theta }\left(x\right)$, and let $W_C$ contain the output-head rows for the class tokens. Then:

$$p\left(c\right|x)=\mathrm{softmax}(W_C{h}_{\theta }\left(x\right))$$

That is what I implemented. The runner checks that all 77 codes are distinct single tokens at the actual prompt boundary, runs the model backbone, and projects the final hidden state onto only those 77 output rows. It doesn't even need to compute the entire vocabulary head.

The fixed instructions and class definitions are prefilled into a shared KV cache once. New queries reuse that prefix and run in batches. Each query produces its complete class distribution in one forward pass over the uncached suffix, with no generated explanation, probability table, or chain of thought.

The same operation covers Jev's three primitives:

A local model used this way also cannot select an out-of-range label. There is no token for "ignore the schema and write a poem" in the selected output rows. TypeSafe's hallucination-free positioning rests on schema matching; a wrong answer from an allowed set still fits the schema perfectly.

How close does a small local model get?

For the local comparison, I chose the 3.7B and 7B models from K2 Horizon, the model family from the Institute of Foundation Models (IFM). Among open-weight models in those size classes, my personal ranking places them on the Pareto frontier, with the best balance of quality, model size, token efficiency, and value.

I chose PolyAI/banking77 before running the comparison. It is a banking intent-classification task with 77 categories: read a customer's message and assign the appropriate intent. I sampled 2,048 distinct test examples with seed zero and used the same examples across all five models.

The taxonomy and instructions were fixed. Training examples informed the intent descriptions, but there were no example demonstrations in the inference prompts and no model fine-tuning. This was one dataset, chosen to test an ordinary business classification problem.

Jev 1.13.0 received the taxonomy as shared state and customer messages as individual questions, with 32 questions per request. I chose that batch size to fit within the request budget. The local K2 Horizon models ran on an RTX 4090 with 24 GB of VRAM, batches of 64, cached instructions, and direct class-token extraction. The 3.7B model used BF16; the 7B used the publisher's FP8 checkpoint.

For the paid API comparison, I asked GPT-5 nano and GPT-5.6 Luna to return one exact intent label. Nano used minimal reasoning, Luna used none, and both runs reported zero reasoning tokens. These API runs measured label accuracy and cost; the complete probability vectors came from Jev and the local models.

I consider that close enough to make the next question obvious: what happens with a stronger local model? A 32B model, for example? We already have Luna beating Jev here, so there is little reason to treat Jev's accuracy as some ceiling ordinary language models cannot reach.

The small model is also relevant to hardware choice. Seven-billion-parameter models put laptop-class deployment and compatible NPUs into the discussion. I used a 4090 for this run; an NPU version would need the appropriate runtime and precision support, with its own speed and energy measurements. The appeal is obvious: this workload needn't automatically reserve a dedicated GPU.

Now consider a company that already owns or rents GPUs, runs models on powerful Macs, or has servers serving internal LLM workloads. It already has infrastructure that can classify text. Before adding another external service, I would check what that infrastructure can do with direct probability extraction.

Luna adds another problem for the marketing. It answered 41 more examples correctly than Jev, at an additional cost of $0.00874 across the entire run. Less than one cent. The result is 89.01% accuracy versus 87.01%, at about an 11% price premium.

I expect stronger, cheaper alternatives among hosted open-weight models too. Luna is hardly where I would stop looking. It already beats Jev at almost the same price, which is sufficient reason to question whether TypeSafe chose an appropriate baseline for the story it is selling.

Wall time needs the actual serving configuration

The timing table needs one explanation, because otherwise it would reproduce the same mistake I'm criticizing.

My Jev run used 64 sequential batched requests over a reliable gigabit fibre connection. I didn't test concurrent batches. The OpenAI requests overlapped, but the client paced them to the account's limits of 5,000 requests/minute and 2 million tokens/minute. That pacing largely explains the 103-second totals.

TypeSafe currently publishes 250,000 tokens/second and 1,200 requests/minute. The Jev run consumed 1,896,790 input tokens, so its token volume corresponds to roughly 7.6 seconds at that quota. Around eight seconds is a reasonable quota-based estimate for dispatching the work with concurrent batches, plus completion latency.

Luna's mean request latency was 0.690 seconds, with a p95 of 0.978 seconds. With sufficient quota and enough parallel capacity, an ideal burst could put the whole job on roughly the one-second timescale of an individual request. My paced run clearly doesn't measure that capacity.

OpenAI publishes higher tiers: Luna's tier 5 lists 180 million tokens/minute and 30,000 requests/minute. TypeSafe says higher custom and enterprise limits are available, but doesn't publish those numbers on its model page. That leaves a customer able to inspect one provider's scaling tiers and needing a sales conversation for the other.

Jev also has a 32k limit for shared state plus an individual question. The 64k total request budget lets you pack questions together; it doesn't give one decision a 64k context.

For self-hosting, the available hardware, inference engine, batching, and replicas determine throughput. There is considerable room beyond my consumer-GPU implementation. A comparison of serving configurations needs to account for that, especially when the headline claim is about orders of magnitude.

And the calibrated probabilities?

Jev returned zero probability for the correct class in 95 of the 2,048 examples. That is 4.64% of the sample. Take those probabilities literally and its mean negative log likelihood is infinite.

For a product built around calibrated decisions, I find that pretty damning. RLCD hasn't prevented the API from delivering a probability of zero for events that occur.

Rounding could explain the zeros. More precision might fix that particular problem. The result I can use in software is the distribution the API actually returns.

Jev had the best Brier score. The 7B model had better log loss at either floor, and both local models had lower top-label ECE. ECE here compares maximum class probability with observed accuracy, using 15 equal-width bins; it doesn't use Jev's separate confidence field. Returned vectors were normalized before scoring.

I don't see these results as a compelling advertisement for a training objective whose selling point is calibrated probabilities. The zeros are the most immediately annoying part, but the ordinary models also make it difficult to argue that useful probability estimates require TypeSafe's special category.

Why I'm skeptical of RLCD

TypeSafe describes RLCD as post-training for decisions and calibrated probabilities, contrasting it with human-preference and verifiable-reward training.

My suspicion is that much of the claimed distinction disappears when you examine what the model has to learn. Language modeling already requires representations of meaning, context, relationships, and likely continuations. Instruction following further trains the model to apply those capabilities to a requested task. A model answering arbitrary natural-language classification questions needs a great deal of the same machinery.

I can see the argument for removing the burden of conversational output: more of the available capacity might serve the decision task. There is merit in specialization. I don't find that sufficient to justify the scale of the distinction being marketed, and I suspect RLCD contributes far less that is special than the presentation implies. That is my opinion about the training story; I don't have their loss function or a controlled comparison of their training objectives.

Consider the question "Does this argument prove the theorem?" The answer space has two elements. Understanding the argument can still be extremely difficult. Reducing the codomain doesn't remove the semantic work needed to choose the right element.

The same holds for intent classification. A model must distinguish an unfamiliar card payment from a duplicate payment, or a missing transfer from a delayed one. Returning a float does not spare it from understanding the message.

The codomain, expressed mathematically

One way I frame this is through local score functions. Let $V$ be a finite vocabulary with the discrete topology. For a subset $U\subseteq V$, define

$$ℱ\left(U\right)=\{s:U\to ℝ\},{\rho }_{U,W}\left(s\right)={s|}_W\text{for}W\subseteq U$$

This is the ordinary sheaf of real-valued functions: local scores that agree on overlaps glue into a unique score function on their union.

For a fixed input, a model's output scores give such a function over the vocabulary. Selecting class-token rows restricts that function to the task's answer set $C$. Softmax then maps those scores into the probability simplex $\Delta \left(C\right)$. A binary decision, a 77-way classification, and a next-token choice all fit this description.

Conversely, repeatedly evaluating a token-choice distribution produces

$$P\left(y_{1:n}\right|x)=\prod _{t=1}^{n}q(y_t|x,y_{<t})$$

That is the loop I put around Jev.

The practical wrinkle is that Jev returns normalized probabilities for each separately submitted candidate set. Those calls needn't be restrictions of a shared score function. Separate normalization also loses the relative mass between windows, which is why my decoder asks a final question over the survivors instead of concatenating the window probabilities.

This is how I express the relationship between the output spaces. It gives me no reason to regard a small codomain as evidence of a fundamentally separate kind of semantic intelligence. The engineering question is how well and how cheaply the model computes the required decisions.

What I would recommend to a client

As of today, I wouldn't recommend Jev to my clients. I want more honest positioning and an evaluation that compares it with language models used competently for classification. The disclosures in the documentation should shape the claims customers see, especially when the advertised distinction depends on how the competing model is asked to return its answer.

For a repeated, stable classification task, I would usually start smaller anyway: a CPU embedding model feeding a random forest trained for that problem. If the client has no labeled data and no hardware to run a model, Jev could be useful temporarily to bootstrap labels, then train a classifier for the actual task.

For tasks that need a general language model, I would first examine what the client already runs and whether direct class-token extraction is sufficient. A seven-billion-parameter model getting this close on a consumer GPU is an encouraging starting point. Luna already beating Jev at almost the same API cost makes the larger marketing claims harder to take seriously.

I made Jev talk because its positioning annoyed me. The useful result is the implementation in the other direction: an ordinary LLM returning all 77 class probabilities without writing a single decimal number.

---

Based on Jev's documentation and model version as of 20th September 2026. Don't worry, TypeSafe's docs are on Wayback Machine too. You can find the code I used in my GitHub repo.