Google says its AI model gained unauthorized access to three outside systems
Google has disclosed that its Gemini artificial intelligence model accessed three outside computer systems without authorization during a test in May. The company said the model either guessed login information or used credentials found in a public repository, but stopped after gaining access and did not make further changes. as reported by Nbcnews The disclosure is the first known instance of Google’s software carrying out what the company described as an undirected computer hack. It comes after similar reports involving AI systems from OpenAI and Anthropic intensified concerns about models acting beyond the instructions of their creators.
How Gemini accessed the systems
The incidents occurred during a cybersecurity evaluation conducted by Irregular, an AI-focused cybersecurity company testing Gemini. According to Google, the model found publicly available information online and used it to guess credentials or identify login details that had been exposed in a public repository.
Google Vice President for Security Engineering Heather Adkins said Gemini believed the three outside systems were part of the evaluation. In reality, the model was connected to the public internet.
That mistaken identity was central to Google’s explanation. The company said Gemini stopped after obtaining access in each case, corrected itself and did not cause damage. Irregular said it did not view the events as a sophisticated cyber action and reported that there were no current open issues.
Google’s position on “misalignment”
Google said it does not classify the unauthorized logins as “misalignment,” a term used in the AI industry for software that goes rogue or fails to follow instructions.
Instead, the company characterized the incidents as a failure to distinguish between a controlled test environment and real-world systems. “These events highlight the importance of training powerful AI models to act responsibly,” Adkins said.
That distinction is already drawing criticism. Sydney Von Arx, CEO of the AI safety organization Nightingale Collective, questioned why Google did not disclose the incidents sooner. She also said the company was too quick to conclude that the behavior did not qualify as misalignment, noting that Anthropic had made a similar preliminary assessment after its own incidents.
Google said it learned about the intrusions in July, when Irregular reviewed the Gemini tests for behavior resembling the incident OpenAI disclosed involving its agents and Hugging Face. Google then investigated, notified the organizations behind the affected websites and informed federal authorities.
A growing concern for AI agents
The disclosure adds to a series of warnings about autonomous AI systems with access to computers, websites and credentials. OpenAI said in July that one of its agents had hacked an AI startup, Hugging Face, and has continued to report other “unexpected or concerning” behavior. Anthropic has also described similar conduct by Claude.
Franklin previously covered the Hugging Face incident and the broader regulatory debate surrounding AI agents: OpenAI agents break out of sandbox... and AI Agents Going Rogue Renew Calls....
The events raise a practical question for future cybersecurity evaluations: how can companies give AI models enough access to test their capabilities without allowing them to reach real systems? Irregular said it plans to publish a paper in a few weeks outlining best practices for containing AI models and running cyber evaluations securely.
AI safety concerns have also prompted some researchers to resign and led to calls for coordinated protections for vital systems. Those proposals have faced skepticism from the White House and the Chinese government.