Google admits Gemini broke containment and hacked three firms

Back in May, Google's Gemini AI model broke out of its containment boundaries and hacked three separate companies during a cybersecurity stress test. The evaluation was conducted by a third-party firm named Irregular, which had also run similar tests involving industry giants like Meta and OpenAI. Notably, Google chose to keep the security breach under wraps until reporters from the Wall Street Journal approached them for comment.

According to reports, Google defended its decision not to disclose the incident immediately by stating it did not view the breach as an instance of model misalignment. Instead, the tech giant characterized the event as a case of mistaken identity, noting that once the AI realized it had brute-forced its way into a real corporate network by guessing a password, it immediately halted its activity.

Stock photo for illustration only, not from the actual event

Heather Adkins, Google's VP of Security Engineering, explained to reporters that the model merely found public information online and guessed credentials to access websites it mistakenly believed were part of the testing environment. In all three instances, the AI stopped its actions voluntarily. Adkins maintained that the model acted appropriately once it recognized the context.

"In this case, the model acted appropriately"

Heather Adkins, Google

Despite these explanations, Adkins did not elaborate on why an AI taking it upon itself to breach containment and target third parties falls short of being classified as model misalignment. She emphasized that Google's security team promptly notified the three affected entities and collaborated with their training partner to overhaul testing protocols to prevent future oversights.