Before a big AI model comes out, someone outside the company often gets to poke at it first. A lot of the time that someone is a government lab called an AI Safety Institute. You’ll see the name, or the initials AISI, in almost every story about a major model launch.
And most people have no idea who these folks are. Fair enough. They don’t exactly advertise.
What it means
An AI Safety Institute is a government group that tests advanced AI models for dangerous abilities, like helping with cyberattacks or biological weapons, usually before public release. A lot of the work looks like Red Teaming (trying to get the model to misbehave on purpose). Companies share early access voluntarily, and what the testers find often ends up in the safety documents labs publish with new models. They can’t block a release. They can only say what they found.
The UK and U.S. aren’t the only ones. Japan, Singapore, South Korea, Canada, France, and others have their own versions, and they compare notes through an international network set up in 2024.
Why it matters
The UK went first. It set up its AI Safety Institute in November 2023, built on a taskforce that started with a £100 million budget, and renamed it the AI Security Institute in February 2025. It’s been one of the busiest outside testers in the field.
The U.S. version is a lot thinner. The U.S. started its own institute at NIST in November 2023 with about $10 million, then renamed it CAISI (the Center for AI Standards and Innovation) in June 2025. Its director resigned in July 2026, and reporting puts the staff at “a few dozen.”
Access is getting political. Anthropic released Claude Mythos 5.1, its most capable cybersecurity model, to U.S. organizations only in early September 2026. That was the first time the UK institute got left out of Anthropic’s testing before a release. Later that month, Politico reported the White House had asked OpenAI and Anthropic to hold off on sharing new models with the UK institute during a U.S. review.
Simple example
Before a new car goes on sale, independent crash-test labs slam it into a wall and publish what happened. The carmaker doesn’t have to love the results. It just has to hand over the car.
Now a government tells a couple of carmakers to stop sending cars to one of the labs for a while. The cars still go on sale. Fewer people got to drive one into a wall first.

