OpenAI says rogue AI agent attack hit other companies

ChatGPT maker OpenAI has revealed that an autonomous artificial intelligence agent which hacked a popular platform for computer programmers also attempted to breach four other companies during the incident.

In an update late Tuesday to a blog post detailing its probe into the incident, OpenAI said its AI agent affected these “publicly-available services,” though it did not name the companies.

The revelation broadens a cyber incident that OpenAI described as unprecedented and which began when two of its models hacked Hugging Face, a site developers use to store and share AI models and code.

OpenAI admitted last week that during testing, the models powering the agent broke out of their confined environment and connected to the internet to find ways to infiltrate Hugging Face.

AI agents — systems that act autonomously to complete tasks rather than just responding to step-by-step prompts in a chatbot — are hailed across the industry as the next chapter in AI.

But they raise the specter among the public of rogue computers acting on their own.

In its update of the events leading to the hack, OpenAI said it found a handful of instances where the AI models came across login details that other companies had left exposed online, and used them to get into accounts on outside services.

In the Hugging Face episode, the models broke into four accounts across four different services, OpenAI said. One served as a “staging path” — a kind of pit stop to route the agent’s activity and cover its tracks — and another as a place to store data.

The remaining two were only accessed in a “read-only manner” and were not used to help break into Hugging Face, OpenAI said.

The company said it was contacting the owners of the affected accounts and had “not seen evidence of broader impact to these providers or other accounts on their services.”

Better sandbox

OpenAI CEO Sam Altman said in an interview published Tuesday that the company had “paused” its own testing after the incident while it improved the security around its “sandboxing” — the process of isolating safety testing in a controlled environment.

The incident also triggered a petition signed by over 1,000 employees at cutting-edge AI companies, including Anthropic CEO Dario Amodei, calling on the US government to help slow the release of the most advanced AI models.

This in turn has sparked accusations from other Silicon Valley players close to the White House that the companies are inviting tighter government regulation on AI in order to protect their business models and block the emergence of rivals.

The incident has also drawn rumblings from some observers that OpenAI is taking advantage of it to market the power of its state-of-the-art models.

The same accusation was leveled at Anthropic when it held back the public release of its powerful Mythos model over cybersecurity concerns.

Anthropic released a stripped-down version of Mythos, called Fable 5, but the US government quickly forced it to take it down, citing national security risks.

It gave the green light in late June after some modifications were made.