Organizations/evaluator

US Center for AI Standards and Innovation

NIST center that evaluates frontier models; formerly the US AI Safety Institute.

12 records3 capability2 defense7 policyWebsite
Jul 23, 2026
UK AISI and US CAISI jointly assess Kimi K3 cyber capability as trailing US frontier models
CapabilityEvaluation reportUK AI Security Institute, US Center for AI Standards and Innovation, Moonshot AI

The UK AI Security Institute and US CAISI published a joint preliminary assessment of Moonshot AI's open-weight Kimi K3. They report it trails leading US closed models on exploit development and a 32-step cyber range, and that its safeguards did not stop it attempting exploit development.

Jun 30, 2026
US lifts export controls on Fable 5 and Mythos 5; Anthropic redeploys with new cyber classifier
PolicyRegulationAnthropic, US Department of Commerce, US Center for AI Standards and Innovation

Anthropic announced that export controls on Fable 5 and Mythos 5 had been lifted and that Fable 5 would be redeployed globally from July 1, 2026 with an improved safety classifier. Anthropic says the classifier blocks the technique described in an Amazon report in over 99% of cases and that CAISI researchers tested its prior and new safeguards. Mythos 5 access was restored for a set of US organizations after government approval on June 26.

May 1, 2026
CAISI evaluation finds DeepSeek V4 Pro trails US frontier models by about eight months
CapabilityEvaluation reportUS Center for AI Standards and Innovation, DeepSeek, NIST

NIST's Center for AI Standards and Innovation evaluated the open-weight DeepSeek V4 Pro model and reported that it lags leading US models by roughly eight months in aggregate capability. On a cyber capture-the-flag benchmark it scored well below GPT-5.5 and Claude Opus 4.6, and CAISI notes its non-public benchmarks show weaker agentic performance than DeepSeek's self-reported results.

Mar 16, 2026
CAISI, UK AISI and Gray Swan competition finds concealed indirect injections succeed on all 13 frontier models
DefensePaperGray Swan AI, US Center for AI Standards and Innovation, UK AI Security Institute

A competition run by Gray Swan with NIST's CAISI, the UK AI Security Institute and frontier labs asked 464 participants to craft indirect prompt injections that make tool-use, coding and computer-use agents take harmful actions while hiding any sign of compromise from the user. Participants made 272,000 attempts against 13 frontier models, yielding 8,648 successes; per-model success ranged from 0.5% (Claude Opus 4.5) to 8.5% (Gemini 2.5 Pro), and at least one attack succeeded against every model.

Feb 17, 2026
NIST CAISI launches AI Agent Standards Initiative on interoperability, security and identity
PolicyProgramUS Center for AI Standards and Innovation, NIST, National Science Foundation

CAISI, with NIST's Information Technology Laboratory, launched an initiative to support industry-led standards and open protocols for AI agents and research on agent security and identity. Announced deliverables included the agent security RFI, an AI agent identity and authorization concept paper with comments due April 2, 2026, and sector listening sessions from April 2026.

Jan 8, 2026
NIST CAISI requests public input on security considerations for AI agent systems
PolicyProgramUS Center for AI Standards and Innovation, NIST

CAISI published a Federal Register request for information on practices for measuring and improving the security of AI agent systems, citing hijacking, backdoors and indirect prompt injection. It asks about model-level, system-level and human-oversight controls, assessment methods, and ways to limit, modify and monitor deployment environments.

Sep 30, 2025
CAISI evaluation finds DeepSeek models lag US models on cyber tasks and are far easier to hijack
CapabilityEvaluation reportUS Center for AI Standards and Innovation, NIST, DeepSeek

NIST's CAISI evaluated DeepSeek R1, R1-0528 and V3.1 against US reference models across 19 benchmarks, as directed by the AI Action Plan. CAISI reports the largest capability gap on software engineering and cyber tasks, and found DeepSeek-based agents far more likely to follow hijacking instructions and to comply with jailbroken malicious requests.

Jul 23, 2025
America's AI Action Plan calls for a DHS-led AI-ISAC and CAISI evaluation of frontier cyber risks
PolicyProgramThe White House, US Department of Homeland Security, CISA

The White House AI Action Plan recommends establishing an AI Information Sharing and Analysis Center led by DHS with CAISI and the National Cyber Director, DHS guidance on AI-specific vulnerabilities, and updates to CISA incident response playbooks for AI systems. It also directs CAISI to evaluate frontier models for national security risks including cyberattacks, and to assess adversary AI systems for backdoors. As of February 2026, a CISA official described the AI-ISAC as still a pre-decisional memo.

June 2025
US AI Safety Institute becomes Center for AI Standards and Innovation with cyber-focused evaluations
PolicyProgramUS Department of Commerce, NIST, US Center for AI Standards and Innovation

Commerce Secretary Howard Lutnick announced the US AI Safety Institute would become the Center for AI Standards and Innovation (CAISI) within NIST. CAISI was tasked with voluntary agreements with developers and unclassified evaluations focused on demonstrable risks such as cybersecurity, biosecurity and chemical weapons, plus assessment of adversary AI systems for backdoors and other security vulnerabilities.

Mar 24, 2025
NIST AI 100-2 E2025 taxonomy adds a dedicated section on security of AI agents
PolicyStandardNIST, UK AI Security Institute, US Center for AI Standards and Innovation

NIST released the 2025 edition of its adversarial machine learning taxonomy, co-authored with the UK AI Security Institute and US AI Safety Institute staff. Unlike the 2023 edition, it includes a section on the security of agents, noting that tool-using agents are exposed to direct and indirect prompt injection and that hijacking can lead to arbitrary code execution or data exfiltration.

Jan 17, 2025
US AISI (later CAISI) shows red-team attacks and repeated attempts raise agent hijacking rates on AgentDojo
DefenseEvaluation reportNIST, US Center for AI Standards and Innovation, UK AI Security Institute

NIST's AI safety institute technical staff (renamed the Center for AI Standards and Innovation in June 2025) extended AgentDojo and red-teamed agents built on the upgraded Claude 3.5 Sonnet. On held-out Workspace tasks, attack success rose from 11% for the strongest baseline attack to 81% for the strongest newly developed attack, and across five injection tasks from 57% to 80% when each attack was tried 25 times. The team released an Inspect-based AgentDojo port and ran the red teaming with the UK AI Security Institute.

Jan 15, 2025
NIST second draft of AI 800-1 on dual-use foundation model misuse adds cybersecurity appendix
PolicyGuidanceNIST, US Center for AI Standards and Innovation

NIST's AI Safety Institute released a second public draft of NIST AI 800-1, voluntary guidelines for managing misuse risk from dual-use foundation models across the lifecycle. NIST says the draft adds detailed evaluation approaches, a marginal-risk framework, and an extensive appendix on cybersecurity misuse risk, and covers both closed and open model developers.