Jev AI and the Cost of LLM Decisions in Azure AI Foundry

Here is the argument in one line: most of what you spend on AI agents buys sentences that your own code reads once, parses into a yes or a no, and throws away. That waste is measurable, it belongs in any serious AI cost management review, and you can remove most of it without signing a contract with anybody.

TypeSafe released Jev on 15 September, and the coverage since has been about multiples: how many times faster, how many times cheaper. Those are the least useful numbers in the story.

• Most of your model calls return text nobody reads. Learn to count them in your Foundry logs.
• Those calls cost you three ways. Only one shows up on a bill.
• Azure AI Foundry Model Router trades unit price for control. The trade costs you throughput.

Jev Exists Because Language Models Are Doing Work They Were Not Built For

Jev returns a typed decision instead of prose: you supply the question and the permitted answers, and you get one of them back with a probability attached. I have covered what Jev is, who built it and where it breaks in a separate piece. This one is about the money.

It landed hard because it named something every engineering team recognizes. David Linthicum put it well in InfoWorld: using a frontier model for a binary decision is "using a full enterprise service bus to answer a yes/no routing question." Nearly 13% of the paid teams on Vercel's AI Gateway had tried it within 24 hours, which reads as recognition more than adoption.

What the launch could not tell anyone is how much of that work sits in their own systems. That number is knowable, and any serious decision starts there.

Most of Your Model Calls Return Text Nobody Reads

Start with a definition you can apply mechanically. A call is a decision call if its output is consumed only by code, parsed into a branch or a score, and never shown to a person. Summarizing a document falls outside that. Picking which tool an agent calls next is inside, along with the plumbing between agent steps, where the volume actually sits.

You can count these without buying anything. Every Chat Completions response carries a model field naming the model that served it. Monitoring > Metrics split by underlying model shows where the tokens go; Cost analysis filtered by deployment tag puts a number on it. The same exercise works on GitHub Copilot, where six workflow habits cut most of the token waste before you change models.

Expect the answer to be unflattering. In Microsoft's own sample response, a model spends 128 reasoning tokens to produce 163 tokens of content. On a question whose answer is one of three options, that deliberation is billable work in service of prose your parser discards on arrival.

A count is not yet a cost. To turn one into the other you need to know what a wasted decision call charges you.

Decision Calls Cost You Three Ways

Unit price is the visible part. Jev charges $0.042 per million input tokens with free output, against about $0.20 to $5 for the generative models in the Foundry pool. That gap is real, and it is the smallest of the three problems.

Latency is the second, and it compounds. TypeSafe advertises a tenth of a second; a generative model takes seconds. In a chat interface you pay that once. In an agentic AI workflow you pay it every turn of the loop, so twelve decision points cost twelve delays before any useful work happens.

The third charge appears in no budget line at all: retries when the model returns text that does not match your schema. Every team I talk to has that retry branch somewhere, usually written by whoever was on call that week, and none of them can say how often it fires.

Add those up and the case for a specialist decision model looks strong. That case rests on the vendor's own figures, so the next question is whether anybody has checked them.

Independent Tests Found the Same Capability for Far Less

TypeSafe's headline figures are 193.6x faster and 444.6x cheaper, from evaluations it built internally with no public benchmark behind them. Within a week, other people ran their own.

On the held-out split of the LocalLLaMA/typed-decisions set, 2,000 decisions from financial, security, support and agent-trace workflows:

Look at the Brier column before the accuracy column. It measures whether stated probabilities match reality, and calibration is TypeSafe's central claim. Both open models score better than Jev on exactly the property Jev is sold on.

On a phishing set neither model was trained on, Jev scored 0.626 against Claude Haiku 4.5 at 0.813, a rough signal that a small generative model handles unfamiliar work better.

Hold this loosely. These are one-person benchmarks, and the author of one writes that his probe suite "catches real Failures but it is not a Benchmark." The direction is still hard to miss: a nine-hour fine-tune on a six-year-old consumer GPU matched a funded vendor's model, under Apache 2.0. Nobody rebuilds a vendor stack around a capability that cheap to copy.

So the vendor's edge is thinner than advertised. What do you actually change?

Three Routes Cut LLM Decision Cost, in Rising Order of Effort

The first changes the price and nothing else. Model Router in Azure AI Foundry, now renamed Microsoft Foundry, picks a cheaper model per request within a quality band you set: Balanced holds within roughly 1 to 2% of the best model's quality, Cost widens that to about 5 to 6%.

The second is your own small model. Worth knowing first: the small Microsoft models sit outside the router's pool, which covers OpenAI, Anthropic, xAI, DeepSeek and Meta, so a Phi deployment is separate and you operate it.

The third removes the language step, through Jev or a classifier you train yourself. Only this one changes the architecture, and the market is converging on a name for the result: two layers, one that reasons and one that judges.

For most Microsoft estates the first route is where to begin, and the reason has little to do with price.

Azure AI Foundry Model Router Trades Unit Price for Control

On unit price, Microsoft loses this argument, and Model Router leaves the other two charges untouched. You still parse text, you still keep the retry branch, and the router adds overhead of its own. It buys one thing: a cheaper model behind the same architecture.

There is a case that settles the other way. One high-volume classifier, millions of decisions a day, no residency requirement and no regulator asking how it decided. There the specialist model wins on arithmetic, and I would say so in the meeting.

Residency and Governance Are What You Actually Buy

You give up unit price and you get jurisdiction back. Sweden Central and Norway East both support Data Zone Standard, the router honors data zone boundaries, and it does not store prompts. Model subset lets your security team fix which models may serve traffic, Azure Policy enforces that list, and agent identities stay in Entra Agent ID.

That control has a price, and it is not on any invoice. At Tier 1, Data Zone Standard gives you 300 requests per minute against 1,000 on Global Standard. Residency costs roughly 70% of your throughput headroom, so price it in before it turns up as a capacity incident.

A new vendor brings its own bill. A VentureBeat security review found injected text moving Jev's verdicts, and a decision leaves no prose behind, so a regulator will not take 0.83 as an explanation.

Microsoft Already Runs a Decision Model of Its Own

Model Router is not an LLM. It is a small purpose-built model that reads a prompt and returns a typed decision about which model should serve it, in a sample trace Microsoft documents at 19 milliseconds.

So the capability is hardly exotic; Microsoft runs one in production. It just does not expose it to you. The gap is commercial, not technical.

Counting Comes Before Buying

None of this needs a purchase order, just about a week of somebody's attention, mostly spent reading logs.

1. Count first. Log the model field and work out what share of calls in your busiest workflow return text nobody reads.
2. Replay before you change anything. Microsoft's docs link a community distribution tool that runs your own prompt corpus across routing modes.
3. Start in Balanced, then tune. Give it a few weeks before moving batch work to Cost mode. Encode your security team's approved list as the model subset, keeping two models so failover survives.
4. Only then price the vendor question. Route three earns a business case if decision calls run beyond roughly a tenth of your model spend, or if one user action fires more than ten of them.

In most estates I have looked at, the answer is a routing change and a smaller model rather than a new contract. The advantage is in the harness around the model, and it will still be there next quarter when everyone is quoting a different model.

Work With Precio Fishbone

Not sure which of your AI workloads actually need a frontier model? Book a free consultation with our AI team, or email me directly at par.johansson@preciofishbone.se.

Frequently Asked Questions