evals & dangerous capabilitiespublic testing

AI Safety Institutes

UK AISI · US CAISI · international network

Government-funded research bodies that test frontier models for dangerous capabilities before and after release, and publish shared evaluation tooling so the work is reproducible outside the labs that build the models.

What they are

State-backed teams set up to evaluate the national-security and public-safety risks of advanced AI. The UK body is the flagship: launched as the AI Safety Institute in late 2023, it was renamed the AI Security Institute in February 2025 — still abbreviated AISI, still inside the Department for Science, Innovation and Technology — sharpening its remit toward cyber-attacks, fraud, CSAM, and chem/bio misuse. The US equivalent, originally the US AI Safety Institute housed at NIST, was restructured in June 2025 into the Center for AI Standards and Innovation (CAISI), still within NIST but reoriented toward standards, security testing, and competitiveness. Both anchor a broader international network of safety institutes (EU, Japan, Singapore, Canada, and others) coordinating methodology.

What they produce

Pre- and post-deployment evaluations of frontier models — measuring capability uplift in cyber-offense, chem/bio, autonomy/agentic behavior, and the effectiveness of model safeguards.
Shared methodology and threat models — turning ad-hoc red-teaming into repeatable, comparable benchmarks across institutes and labs.
Inspect — an open-source framework (UK AISI with Meridian Labs) for building LLM evaluations: datasets, tools, agents, and scorers, with 200+ pre-built evals. See inspect.aisi.org.uk ↗.

Why they matter

They put independent, public-interest evaluation outside the labs that ship the models — a credibility check that voluntary self-reporting can't provide on its own. As an institutional layer they feed emerging governance: model-access agreements with frontier labs, the international safety-institute network, and the testing/transparency expectations forming around regimes like the EU AI Act. Inspect being open-source means the same evals can be run by researchers, auditors, and other governments — not just the institutes.