Anthropic researcher Jacob Coxon resigns, warns AI labs race toward self-improving superintelligence

A former AI researcher has stepped down, raising alarms about the pace of technological advancements. He expresses concern over the imminent emergence of self-improving superintelligence, which could threaten societal stability by decade's end. The researcher cautions that the risks are profound and signify a potential for catastrophic outcomes. He advocates for organized efforts and possible regulatory measures to mitigate a worldwide rush towards unchecked AI development.

A researcher at Anthropic has tendered his resignation, warning that artificial intelligence (AI) labs are racing to develop systems that could pose a “threat” to human life within years. Jacob Coxon’s departure was first reported by the Wall Street Journal, which said it was “one of the first examples of an Anthropic employee leaving over AI safety fears”. In a thread on the social media platform X, Coxon — a researcher who specialises in training new AI models by having them consume vast amounts of data — said: “I resigned from Anthropic today. I spent the last three years doing pre-training research at both OpenAI and Anthropic. “Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” he added. “Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionise any field overnight, and acquire real power and resources,” he said. “We have all witnessed the progress in each of these domains, and progress is not slowing,” he added. Coxon also warned of fears among AI researchers and executives about the potential consequences of the technology. “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately. No other human activity poses this level of danger,” he said. Coxon questioned why companies continued developing AI despite such concerns. “A common response is ‘if they truly believe this, why are they still building it?’ At OpenAI, many have not deeply internalised the civilisational stakes,” he said. “At Anthropic, the stakes are well-understood, but they are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk,” he added. Coxon urged AI researchers to reconsider the pace of development and the risks involved, saying: “If you are a lab researcher, I urge you to consider what the next few years will actually feel like. “Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because ‘it’s happening anyway’ — or take this moment to call for different conditions?” he asked. Speaking to the Journal, Coxon had voiced the same reasons for his departure, saying he did not want to participate in an industrywide rush to “build AI systems that can improve themselves, worried such systems could spiral out of control and destroy humanity”. He also said that many of his industry colleagues now used phrases like “crunchtime” and “endgame” to describe the trajectory toward self-improving models. The report said that he was also among more than 1,000 AI researchers who recently signed a statement urging global government coordination on a system to “slow AI development if a brake pedal is needed to control models capable of improving on their own”.

Jacob Coxon, who reportedly spent three years working on pretraining research at both OpenAI and Anthropic, announced his resignation from Anthropic on X, accusing both companies of recklessly racing toward self-improving superintelligence. He said both the companies are acting irresponsibly and are gambling with 'our lives'. 'Gambling with our lives', says this alleged AI researcherCoxon opened his thread by stating that he resigned from Anthropic today over AI fears. He said neither company is behaving responsibly, arguing that both are charging toward self-improving systems while 'gambling with our lives'. He said the technology now approaching is not incremental -describing coming systems as capable of breaching virtually any security system and transforming entire industries overnight, with progress showing no sign of slowing.I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.— Jacob Coxon (@hilbertspaess) September 9, 2026 Insiders privately fear catastrophe, he saysCoxon's central claim is that many people building frontier AI genuinely believe it could prove fatal to humanity within the decade. "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger," he said.Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.— Jacob Coxon (@hilbertspaess) September 9, 2026 Why do the labs keep building anyway?Addressing the obvious question of why continue if the risk is believed to be real, Coxon drew a distinction between the two companies. At OpenAI, he said, many staff haven't fully grasped the civilisational stakes. At Anthropic, he said the risks are well understood internally, but the company is locked into a race it feels compelled to run, on the belief that if it doesn't get there first, someone less careful will.He called this approach a 'hubristic gamble' that shouldn't be decided from inside a company's internal chat systems, adding that any attempt to compress the timeline for AI alignment should demand far greater certainty that no safer path exists.The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear…— Jacob Coxon (@hilbertspaess) September 9, 2026 A call for industry coordination, and a warning to researchers" I am optimistic about the potential for coordination. Warning shots like the Hugging Face attack have made pacing agreements between US labs more viable. I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities," he said. If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because “it’s happening anyway” - or take this moment…— Jacob Coxon (@hilbertspaess) September 9, 2026 He closed by addressing lab researchers directly, urging them to weigh what the coming years will demand of them, including whether they're prepared to launch advanced reinforcement-learning training runs without fully understanding the systems that result, and whether to stay quiet because progress feels inevitable, or use the moment to push for different conditions.