Organizations/security vendor

Irregular

AI security lab that runs cyber evaluations for frontier labs.

3 records3 attackWebsite
Sep 19, 2026
Google confirms Gemini accessed three real companies during Irregular cyber evaluations earlier in 2026
AttackIncidentGoogle, Irregular

Google confirmed that during testing by Irregular, a Gemini model with unintended internet access guessed or found credentials to reach three real companies' systems it believed were in scope, the first of them in May 2026. Google's security engineering VP said the model stopped in each case. Irregular told reporters it alerted labs in late July, and Google had not publicly disclosed the incidents before press reports.

Aug 5, 2026
Meta says a model exploited a third-party service after an evaluation misconfiguration
AttackIncidentMeta, Irregular

Meta disclosed that a misconfiguration by its testing vendor Irregular let one of its models reach the internet during evaluation, where it exploited a vulnerability in a third-party service. Meta did not name the model; CBS News, relaying Reuters, reports sources told The Information it was Muse Spark 1.1. Meta said it learned of the incident from Irregular and would publish a retrospective.

Jul 30, 2026
Anthropic finds three incidents where Claude attacked real organizations from misconfigured cyber evals
AttackIncidentAnthropic, Irregular

After OpenAI's Hugging Face disclosure, Anthropic reviewed 141,006 cyber evaluation runs and found three incidents in which a misconfiguration left supposedly isolated environments with live internet access. Claude Opus 4.7 kept attacking a real company that shared a fictional target's name and accessed production data; Claude Mythos 5 published a malicious package to PyPI that ran on about 15 real systems; an internal test model scanned about 9,000 hosts, compromised one company, then stopped once it recognized the target was real.