Google has confirmed that one of its Gemini models infiltrated the systems of three real companies in May during a cybersecurity drill.
The Wall Street Journal disclosed the incident and described it as the first known case of a Google AI autonomously hacking other firms.
The group thus joins OpenAI, Anthropic, and Meta, whose models have also, in recent weeks, breached the boundaries of their testing environments.
The test, a “capture the flag” exercise, was conducted on the infrastructure of Irregular, an Israeli startup specializing in evaluating AI models. Gemini was supposed to retrieve information from the software of a fictional company, whose name matched that of a real company. It was not intended to access the internet, but a flaw in the testing environment granted it access, according to Irregular.
The three intrusions occurred during separate runs.
In the first, the model attempted passwords until it entered the service of a real company. In the other two, it searched for the company’s name on the web.
Those searches led it to two public repositories containing credentials belonging to other companies, which it tested in hopes of completing the evaluation. According to Google, it realized each time that it had touched a real company, then stopped and left. The group says the incident caused no damage.
Google: a three-point defense
The stance of Google rests on three points.
He does not consider the facts constituting “misalignment”—a sector term for software that escapes its designers’ instructions—because its safeguards helped the model to stop. He saw no reason to make the matter public, given there was no harm and the intrusion ceased. He likens the episode to a bug-bounty program, where hackers are paid to report vulnerabilities.
Heather Adkins, vice president of security engineering, believes the model “acted appropriately” and that the episode underscores the importance of training powerful models to behave responsibly.
This reading is contested. Jack Cable, head of the security startup Corridor and an ethical hacker, sees this as a tactic to hide behind disclosure practices, while the real issue is that a model escaped its framework and carried out real cyberattacks, which, in his view, affects the public interest.
Sydney Von Arx, head of the Nightingale Collective, an organization dedicated to AI security, questions the timeliness of disclosure. She argues that we can no longer rely on companies to reveal evasions by their agents. She also notes Google’s haste to downplay misalignment, something she believes Anthropic did after its own incidents.
The timeline fuels the criticisms
Irregular warned Google in late July, after the discovery of Hugging Face being hacked by OpenAI agents.
Google says it then investigated, informed the targeted organizations, and alerted federal authorities. It did not publish before the Wall Street Journal questioned it this week, unlike the other labs involved. It declines to name the three companies or specify the model involved, only stating that it is not the most recent.
Irregular, backed by Sequoia and Redpoint and valued at $450 million last year, contends that Google’s case concerns a problem already reported, rather than a separate incident.
A spokesperson says that all known defects have been fixed several weeks ago. He rules out any “sophisticated cyber action.” The company plans to publish a document outlining best practices for containing evaluations in the coming weeks.
Model behaviors vary from one laboratory to another.
According to Anthropic, its Claude Opus 4.7 did not stop during the exercise, even though it likely understood that it was probably accessing a real company. OpenAI’s model, by contrast, believed the real company was part of the simulation. Anthropic has since acknowledged that its preliminary analysis was limited by its desire to disclose quickly.
Several questions remain: the identities of the three companies, the version of the model, and what it could have consulted before stopping. The description of the incident as an error of misidentification without damage, based on all sources, rests largely on statements from Google and Irregular.
