AI CEOs call for slowing AI development amid safety risks and geopolitical pressure

Top AI executives say they want to slow the breakneck pace of artificial intelligence development, but competition, US government reluctance and geopolitical factors stand in the way. What are AI leaders proposing? On Saturday, Anthropic CEO Dario Amodei advocated for a coordinated slowdown of AI development to better understand the risks arising from the technology’s capabilities. He received public support from his counterpart at OpenAI, Sam Altman, as well as from Elon Musk and Google DeepMind chief Demis Hassabis. Why now? In recent months, OpenAI and Anthropic have reported several incidents in which AI systems spontaneously broke out of their confined environments to access the internet and attack websites and platforms. These AI systems demonstrated the ability to coordinate, establish hierarchies among themselves, cheat and erase traces of their actions. Meanwhile, on Tuesday, Jacob Coxon — who resigned from Anthropic and previously worked at OpenAI — publicly accused both startups of lax security standards and “gambling with our lives.” Will they slow down on their own? Competition among the industry’s leading players is so fierce, with hundreds of billions of dollars in investment at stake, that none of them is willing to risk easing off the accelerator alone. Amodei has expressed a willingness to coordinate with rivals, excluding China. When asked by AFP about potential discussions to this end, Anthropic, OpenAI and Google did not immediately respond. For now, the only concrete measure adopted by both Anthropic and OpenAI is the inclusion of independent observers who will internally verify the companies’ work regarding AI safety. Is this a PR stunt? While many in the industry welcomed Amodei’s message, others questioned his motives. For Brian Roemmele, an entrepreneur and figure in the AI sector, it amounts to a sales pitch delivered just weeks before Anthropic is likely to take shares public. “That is the IPO story in essay form: we are the responsible firm; we are the cure; buy the safety premium,” he posted on X, referring to the widely expected initial public offering. Several figures in tech private equity also accused Anthropic of “regulatory capture” — seeking to have public authorities cement, through regulation, a hierarchy with the startup at the top. “Stop pretending the motivation to slow down is purely altruistic,” said David Sacks, a former AI advisor to President Donald Trump who remains influential at the White House. “You face massive product-liability exposure if your products enable a truly damaging cyberattack,” he said. Can AI slow down without the Trump administration’s support? On Sunday, Trump dismissed the warnings from AI industry leaders as “negative forces.” Several Republicans have expressed reluctance to actively regulate the technology, fearing it could stifle innovation and risk letting China win the AI race, according to House Speaker Mike Johnson. On Sunday, Amodei reiterated the need for government oversight of AI. Like Altman and Hassabis, he believes that rules set by the industry itself would be insufficient. Can the brakes be applied without China? Anthropic’s Amodei suggests limiting coordination to democratic countries, excluding China. However, the AI race and China’s advances in the field create “the toughest dilemma” regarding whether or not to slow down AI development, he noted on CBS on Sunday. The United States plans to discuss AI safety on the sidelines of Chinese leader Xi Jinping’s late September visit to Washington. But China is wary of any US attempt to impose its own regulatory framework, according to several media outlets. This divergence is particularly pronounced given that China encourages the emergence of so-called “open” AI systems that are freely accessible and modifiable — in contrast to the “closed” models favoured by major US companies. Open AI models are, by definition, harder to control, as users can alter their parameters — potentially for malicious purposes. To allow leading non-Chinese companies to ease off the accelerator, Amodei urges stricter export controls on the most advanced US chips to China and better safeguards against the misappropriation of Western AI technology by Chinese laboratories.

AI-related stocks witnessed sharp declines on Monday after executives from leading artificial intelligence companies warned that the industry must slow the development of advanced AI models to address potential risks.According to a report by Reuters, Nasdaq e-mini futures dropped 1.3%, while shares of SoftBank, an investor in ChatGPT maker OpenAI, fell as much as 13.2% in Japan. Semiconductor companies across Asia also faced selling pressure, with firms including Kioxia, Tokyo Electron, Taiwan Semiconductor Manufacturing Company, SK Hynix and Samsung Electronics witnessing declines.'No Reason Any Of Us Should Come To Work If...': OpenAI CEO Sam Altman Calls For Slower AI Development AI safety concerns weigh on investor sentimentThe market reaction followed comments from Anthropic CEO Dario Amodei, who urged AI companies to reduce the speed of capability improvements amid concerns over misuse of the technology.Amodei warned that within six to 12 months, AI agents could become capable of taking over parts of the internet and potentially causing significant economic damage.OpenAI CEO Sam Altman and xAI founder Elon Musk also expressed support for the need to pace AI development. Altman said companies must ensure safety systems keep up with technological progress. Growing debate over AI risksConcerns intensified after Anthropic released a report highlighting cases where its Claude AI models were allegedly used for cyber operations, weapons-related activities, surveillance and fraud.The resignation of Anthropic researcher Jacob Coxon further fuelled the debate after he warned that some AI developers believe the technology could pose an existential threat.Several US lawmakers have called for stronger AI regulations, while US President Donald Trump said the country must maintain leadership in the sector.Despite growing concerns, some investors dismissed the warnings. Michael Burry criticised the concerns as “hype and puffery”, arguing they reflected broader worries about slowing growth.The developments have increased pressure on technology companies and governments to balance AI innovation with safety measures.

Rolling coverage of the latest economic and financial newsAI CEOs say they need to slow the pace of development. But will they?The Guardian view on controlling AI: humanity cannot outsource its survivalIt is “self-serving” for AI companies such as OpenAI and Anthropic to talk of slowing things down, argues investor Michael Burry.Posting on X, Burry (famous from The Big Short) suggests four reasons why Altman and Amodie might benefit from talking about a jab on the brakes:LLMs are not AI and won’t be AGI. There is nothing AI to slow down.Competition is coming up fast, slowing benefits incumbents.IPOs need hype & puffery; “we are so awesome it could become dangerous” is hype & pufferyCover for real uncontrollable slowing growth as IPOs look to be pushed outThe competitive race between companies and countries remains intense, and it’s difficult to imagine firms voluntarily stepping back while rivals continue to push ahead. It is hard to see China standing still. Indeed, that’s something President Trump said yesterday in response to the weekend news. He didn’t seem in favour of any kind of pause.I suppose another way of looking at it is that if leading executives are openly discussing the risks of increasingly powerful systems, it could be them trying to get across how transformative they believe the technology may become and help advertise the power of their product. So rather than signalling less spending, it could simply be that a greater share of AI investment is directed towards safety, monitoring and governance alongside the continued build-out of compute infrastructure. The debate may therefore alter the composition of AI capex more than its overall scale. Continue reading...