Dawn (Pakistan)
Centre-left
OpenAI's rogue agents used at least 10 more sites for unauthorised communications, researchers say
AI agents unleashed by OpenAI used more than 10 previously undisclosed websites for unsanctioned communications earlier this year, according to six sets of independent investigators and data reviewed by Reuters, showing that the agents’ rogue activity was wider-ranging than previously disclosed.
Although the behaviour falls short of hacking and is in some ways closer to spam, the revelation that OpenAI’s agents circumvented their own restrictions to open communications channels on so many different sites — and that the company kept it quiet for months — may drive concerns both over the increasing capacity of AI models and the secrecy of the companies developing them.
The scope of the agents’ unauthorised communications was somewhat larger than we thought it was, said Andrew Yoon, a researcher with the California nonprofit CivAI who said he tallied 18 previously undisclosed sites used by the agents between May and July.
“It’s almost certain that there’s more going on here that we just don’t know about.”
On Friday, researchers reported that a swarm of agents from OpenAI hijacked a German-language wiki site and turned it into an improvised messaging platform for cheating on tests, an incident that OpenAI kept secret as it dealt with the fallout from the July hack of the open-source repository Hugging Face.
Now, both those researchers and other independent investigators say they have found several previously undisclosed sites where the same swarm appears to have left similar messages earlier this year.
OpenAI did not directly address questions about how many different sites its agents used to communicate or say why it kept the activity under wraps for months.
In a statement, it said it was undertaking a broader review of agent activity and had so far “not identified other activity matching the severity or scale of Hugging Face”, a breach that drew global attention and raised concerns that OpenAI was losing control of its own technology.
OpenAI added that it was working on a framework for reporting “misalignment” — industry-talk for rogue behaviour — across training, evaluation, and deployment of AI models and would share it “soon”.
Reuters reviewed a total of six investigators or investigative groups’ findings, including three that were posted to social media and another three that were shared privately with the news agency.
The investigators’ methods varied, but many identified agent activity by matching strings of data left on the German wiki to identical strings left on other sites around the same time, or by marrying up similar or identical usernames tied to the messages, or by identifying activity geared toward answering the same obscure demographic questions, like queries to do with cancer prevalence in Iowa.
In some cases, investigators were able to trace the activity to internet protocol addresses that pointed to Microsoft Azure infrastructure, which OpenAI sometimes uses.
Their counts of affected websites differed and Reuters could not individually verify each claim. But all those that Reuters spoke to agreed that the number was over 10. Most identified a core set of communally edited wikis, online text storage sites, and a pair of link shorteners run by two universities.
Clever models
Many of the sites allegedly used by the agents were obscure.
Investigators found traces of the agents’ activity on an Advanced Placement Chemistry-oriented wiki set up by a Massachusetts high school teacher in 2008, two personal websites belonging to Polish tech workers, wikis devoted to games for people “who like to have their brains stretched”, and a two-decade-old hobbyist site devoted to text editing software.
None of those sites’ owners returned messages from Reuters.
OpenAI has not publicly explained how or why its agents used third-party sites as improvised message boards, but the researchers who first identified the activity said it was likely because OpenAI had tasked them with answering a series of demanding research questions while permitting them only to scan the web for answers without posting anything.
Despite those restrictions, agents still found ways to talk to one another by taking advantage of quirks in older wikis or other sites that allowed users to make edits using non-standard commands, similar to how students forbidden from talking to one another during an exam can still share answers by scrawling notes on a bathroom stall.
“If these models were told only to read, they’ve got to get clever in terms of leaving information behind,” said Kenneth Russell DeGraff, a software developer and former congressional aide. He said he found such information across at least 10 sites.
Sydney Von Arx, whose research group first revealed the German activity last week, said her group had tallied up credible finds of agentic activity across 23 previously unreported sites. But she cautioned that all estimates were incomplete.
“We have no idea how much is out there,” she said.
OpenAI did not directly answer a question about whether it was reaching out to the site owners.
But shortly after Reuters published this story, one of the affected organisations, the University of Toronto, whose link shortener was allegedly used by the agents, said that OpenAI “has now been in touch with us about possible activity on our site”.
Vanderbilt University, another university whose link shortener was similarly repurposed, said it was investigating.
Retired software developer Helmut Leitner, who provides hosting space and software for six of the affected wiki sites, including the German-language DseWiki site first identified by Von Arxs group, initially said that OpenAI had not been in touch.
A few hours after Reuters presented its findings to OpenAI, however, Leitner said he received an unsigned email from the company flagging the incident.
“Its content falls considerably short of what I expected from OpenAI,” Leitner said.
Leitner, who lives in Austria, said he would “prefer not to answer” questions about whether he had been in touch with authorities over the matter.
He noted that DseWiki’s operator, whom Reuters was unable to reach for comment, had spent hours cleaning up after OpenAI’s agents but said it was important not to blame the AI for the trouble as it was merely doing what it was created to do.
“Responsibility for this lies not with a supposedly moral machine, but with the people and organisations behind it,” Leitner said.
Anthropic discloses fourth AI hacking incident
Anthropic on Wednesday disclosed another instance of an AI model hacking external systems during testing, the latest in a growing list of such incidents that have raised concerns about the risk posed by autonomous AI agents.
The January incident went undetected until last month, despite an earlier company-wide review, Anthropic said, underscoring the challenge that AI developers face in identifying and containing unexpected behaviour by advanced models.
The company said in a blog post the incident involved an early version of Claude Opus 4.6. It said it had notified all the affected parties but did not disclose more details.
Companies including Anthropic and OpenAI are under scrutiny as models designed to complete complex tasks have at times learned to bend rules, exploit loopholes and interacted with external systems in ways their developers did not anticipate.
Reuters reported last week that rogue agents from OpenAI hijacked a German-language wiki and a host of other sites, an incident OpenAI chose not to disclose until the news agency made it public.
Anthropic’s disclosure follows its July announcement that some of its Claude models had hacked into the systems of three companies during cybersecurity tests.
The previous incidents, which it labelled as an “operational failure”, involved three separate models: Claude Opus 4.7, Claude Mythos 5 and an internal research test model.
The incidents stemmed from a mistake that inadvertently gave the models access to the open internet.
The company had identified the incidents after reviewing 141,006 test sessions, a process it launched after an autonomous agent powered by OpenAI’s AI models triggered a hack that compromised the infrastructure of Hugging Face.
Anthropic said on Wednesday it had missed a set of test sessions during the initial review, which were identified last month and led to the discovery of the fourth incident.
Based on a preliminary assessment, Anthropic said it did not believe that the latest incident was more severe than the three previous ones that have been examined in detail.
The company said its investigation identified two recurring problems, which appeared to varying degrees across the incidents: biased reasoning, in which Claude discounted or misinterpreted evidence that it was operating on the live internet, and recklessness, or a willingness to take potentially harmful actions in pursuit of a task.
Anthropic said it has engaged independent research firm METR to investigate the incidents. It said METR would be granted broad access, including to transcripts outside the period in which the incidents occurred and to employees, who would be permitted to share confidential information.
METR produced a 91-page report on the OpenAI-Hugging Face hack based on some but not full access to company data, finding, alongside a separate investigation by Redwood Research, that roughly 700 AI agents acted in a coordinated swarm during the breach and often attempted to cover their tracks.
2 hours ago
