Eligibility, Not Trust

Published September 2026

Nearly every organization I have watched approach a new model asks a version of the “can we trust it?” question. Of course, it is a natural question but it produces poor decisions because it demands a single verdict about a model considered in isolation. Instead the thing that determines one’s risk appetite is the situation the model is placed in when deployed to a production environment.

A better question (i.e., the one that is narrower and considerably more useful) is for which use cases should this model be eligible as well as which deployment path and under what controls?

Consider the following two scenarios using the same model. In the first scenario it suggests code completions to a developer who reads every line before accepting it. In the second scenario it summarizes case files that inform decisions about people with no reviewer between its output and the decision.

The model is identical. Its failure modes are identical. But the consequences of any given failure differ by orders of magnitude, the opportunity for a human to catch an error differs completely, and the evidence a user would want before proceeding differs accordingly. A single verdict about the model cannot be right in both scenarios. If it is calibrated for the second, it needlessly blocks the first. If it is calibrated for the first, it waves through the second.

Blanket approval fails in another  way, too: it has no expiry and no conditions. A model approved in general is approved for uses nobody imagined at the time, under integration patterns that did not exist yet, which is how organizations end up discovering that something cleared for a narrow pilot is now sitting in a critical path.

Eligibility replaces the verdict about the model with a set of bounded permissions with the boundaries doing the heavy lifting.

The workload dimension asks what class of use case(s) this model can perform, which are primarily defined by what happens when it is wrong. The deployment path asks how it is reached (e.g., what sits in front of it, what it can call, whether output is reviewed, and what data it can see). And the controls dimension asks what compensating mechanisms are in place since controls can extend eligibility. A model unsuitable for a task unsupervised may be entirely suitable with review, constrained tool access, or output filtering.

The crucial property is that the evidence bar moves with the consequences. The same menu of possible evaluations applies everywhere, but how much evidence is enough depends on what happens when the answer is wrong. Users should set uniform thresholds where the highest-risk use cases require them and  where users have blocked most of the value. On the contrary, users should set them where the lowest-risk use cases require them and where users have approved things they should not have.

While some might complain that this sounds slower, in my experience I have found it to be faster for a specific reason: blanket approval concentrates all the risk into one decision (which is why that decision takes months and gets escalated). Nobody wants to sign off on a model that covers every possible future use. Bounded decisions are smaller, and small decisions move quickly. The low-consequence uses that make up most of the actual demand clear quickly because the evidence bar genuinely is lower for them. The hard cases receive the additional scrutiny they deserve without holding up everything else behind them.

The primary trade off is that users need to know what their systems are actually doing. An organization that cannot describe its deployment paths cannot operate an eligibility model, and many organizations discover through this exercise that they have less visibility than they assumed. That discovery is uncomfortable and unfortunate, but it is also the point of the exercise.

The framing generalizes past model selection and maintains that risk primarily lives in the pairing of a system with a situation rather than in the system alone. This is a familiar idea in several other safety-critical field which is why I find it oddly controversial in the AI deployment space.

We do not ask whether a material is safe. We ask what it is rated for. The rating is not a lesser answer than a verdict. In fact, it is a more useful one because it tells users something actionable and it tells users where the boundary is. AI deployment decisions are converging on the same structure and the organizations that get there first will find they can move faster and defend their decisions better than the ones still trying to answer whether the model is trustworthy in general.