There is no shortage of evidence anymore. Models are evaluated constantly. Red teams probe for failures nobody anticipated. Benchmarks appear faster than anyone can keep up with. Producing evidence is no longer the hard part. Understanding what it means, and whether it is strong enough to support a decision, is.

Much of my work sits in the space between measurement and decision-making. Other teams run evaluations, conduct experiments, and generate evidence. I spend my time looking across those different sources, understanding what each one does and does not tell us, and determining what conclusions can reasonably be drawn from them.

A result may be statistically significant and still be a weak basis for action. Another may be imperfect but point to a genuine risk. Most of the judgment lies in distinguishing between the two.

The technical evidence is only part of the story. The questions that interest me most are often the ones that emerge after the metrics have been calculated: who benefits from a system, who bears the risk when it fails, what assumptions are being embedded into institutions, and what dependencies are created when organizations build on technologies they do not control.

A finding can be technically correct and still support a bad decision. Likewise, a system can perform well against its evaluations while creating risks those evaluations were never designed to measure. Understanding the difference is a large part of what I do.

Recently, much of that work has focused on open-weight AI models, where the technical and institutional questions become difficult to separate. Evaluating capabilities, safeguards, and failure modes is important, but so is understanding what organizations are actually acquiring when they adopt someone else’s model, what they become dependent on over time, and what risks are transferred rather than eliminated.

Those questions are often debated extensively but tested surprisingly little. I am interested in bringing evidence to questions that are usually argued from intuition, ideology, or optimism.

At its core, my work is about helping people avoid becoming more certain than the evidence allows.