Most of my work is figuring out whether an AI system is ready for people to use, and what needs to change if it isn't.
At Microsoft, I help turn AI research into products people can use and models others can build on. My work spans consumer AI, AI agents, foundation-model releases, deployment decisions and the internal workflows that support them, including building a new function as a division took shape.
As a Principal Technical Program Manager, I work with researchers and engineers to define requirements, test and evaluate how systems behave, and resolve gaps before release. I connect model behavior to the experience a team is trying to deliver, then help translate those findings into product changes and release decisions.
I use statistics to understand capabilities and uncertainty, and I look closely at the incentives, institutions and dependencies shaping adoption. A model's performance matters alongside who can adapt it, what deployment requires and how an organization can change course.
My current work focuses on open-weight models and their ecosystems. I bring together use cases, evaluation findings and remediation options, analyze the evidence, and advise on deployment choices. That includes the commercial and geopolitical implications of building on a particular ecosystem, as well as what an organization can actually inspect or adapt. I co-led the development of a joint Microsoft and NVIDIA framework, published in October 2026, that helps enterprises identify the risks of deploying open-weight models and match them to evaluations and mitigations. It's available on GitHub.
I spend just as much of my time on AI agents, which act on people's behalf with real tools and permissions, often without anyone watching each step. What I keep coming back to is how control over these systems can erode one sensible delegation at a time, and how anyone would know whether the checks around them still hold.
I'm also exploring world models and their potential to support creation, learning and planning. I'm interested in how simulated experience could help people anticipate consequences, and in what it would take to test a model's predictions before relying on them in a real task.
Writing
Control Is an Empirical Claim
Losing control of AI systems probably won't happen all at once. It's more likely to creep in as we give them more access, which is why the checks around them need regular testing by people trying to get past them.
Openness Is a Safety Property
I argue that being able to fix an open-weight model, legally and in practice, matters as much for safety as who can download it, especially for the companies deploying it.
World Models and the Limits of Simulated Experience
World models could let AI systems try things out in simulation before acting in the real world. I'm interested in how the people relying on those simulations would know when the predictions are wrong.
On the Subtle Differences within Model Censorship
When a model won't answer, the information might be missing from its training data, learned in a distorted form, or known but held back because of how the model was trained. From the outside these look the same, even though each one calls for a different fix.
Owning the Infrastructure Doesn't Necessarily Mitigate Model Risks
Running a model on your own servers, or in your own country, gives you control over the system around it. It doesn't tell you much about how the model itself will behave, which is a separate question that still needs testing.
Eligibility, Not Trust
Asking whether a model can be trusted is too broad to answer. I think the more useful question is which uses a model is suitable for, and the more serious the use, the more evidence you should need.
Publications
Most of my assessment work stays internal and confidential, because it shapes product and release decisions before anything ships. Some of it has also led to published work.
Risk Management for Open-Weight Models in Enterprise
Microsoft and NVIDIA, 2026. A shared risk taxonomy, a map of evaluations to those risks and a process for acting on the results. I co-led its development and editing.
Phi-3 Safety Post-Training: Aligning Language Models with a “Break-Fix” Cycle
Microsoft, arXiv, 2024. How repeated rounds of red teaming and safety post-training were used to align the Phi-3 models. Co-author.
Selected work
Research-to-product delivery
Microsoft
I've led model-behavior requirements, testing and release-readiness work across Microsoft's foundation-model releases and AI product integrations, from Copilot Voice and Vision to agents like Copilot Cowork and Autopilot. My hands-on testing goes back to adversarial prompting of early GPT-4, and for Phi-3 I embedded with Microsoft's AI Red Team to develop their break-fix cycle. With research and engineering teams, I turn test findings into feedback and product changes for the intended experience.
Model and ecosystem strategy
Microsoft
I lead work to consolidate use cases, evaluation findings and remediation options for open-weight models. Drawing on that evidence, I advise Microsoft executives on deployment and ecosystem choices. That advice connects model limitations and practical use cases to commercial fit, sovereignty concerns and the geopolitical dependencies an organization takes on.
Building new functions and workflows
Microsoft
When Microsoft AI formed in 2024, I built its Responsible AI program from zero, working with Mustafa Suleyman and his startup-origin research team from Inflection as they began shipping products at Microsoft's scale. I also worked directly with developers and engineering users to define requirements and test the migration of an internal review workflow, resolve edge cases and retire legacy dependencies without disrupting active cases.
Applied data science
World Economic Forum
Earlier, I led data science at the World Economic Forum, using machine learning and natural language processing across the Forum's global stakeholder network to separate genuine early signal from noise. That work became the analytical layer behind Transformation Maps, available through its Strategic Intelligence platform.
Geospatial data visualization
Carnegie Mellon University CREATE Lab
Alongside my work at the Forum, I collaborated with Carnegie Mellon University's CREATE Lab on EarthTime, turning geospatial and climate-projection data into stories that show the results of past decisions through time.
Contact
If you're working on AI products, research or assessment, I'd welcome a conversation. andrew@andrewberkley.com