Amodei, Altman and Musk back “pacing the frontier” with independent AI evaluators

Amodei, Altman and Musk agree on one thing: slow the frontier down A thread went up on r/ArtificialInteligence tonight with a title that would have been a joke a year ago: three of AI's biggest rivals agree on something. The something is that the technology is moving too fast. The thread had no comments yet when I read it, so here is the story behind it, and what I take from it as someone who runs coding agents all day. What happened On Saturday Dario Amodei published an essay called We Must Pace the Frontier. The line everyone is quoting: "We must slow the pace at which we improve the capabilities of AI models." He adds that progress will still seem fast. The proposal is about the rate of capability gains, not a halt. Within hours Sam Altman wrote "I agree with Dario that we need to pace the frontier" and said OpenAI would take on outside evaluators with employee-like access. Elon Musk posted three words: "Dario is right." OpenAI also said it will not go public this year. The Washington Post, CNBC and Axios all ran it the same day. The three steps Embedded evaluators. Every frontier lab gives a third-party team (Amodei names METR) ongoing, employee-like access: desks, badges, company laptops, the same permissions as the internal risk team, and the right to publish findings without editorial control by the company. Anthropic is doing this on its own, starting now. Common standards among the labs in democracies, with government cover for the antitrust problem of competitors sitting in a room agreeing to slow down. Pacing is tied to observable capabilities: if a model can do X, it ships with certifications of alignment properties Y and Z. Agreements with authoritarian governments, in four escalating levels: ban AI for bioweapons, mutual pre-release testing, speed limits on recursive self-improvement, and a full pacing agreement, which he calls unlikely. Why now Two reasons in the essay. The first is recursive self-improvement, which Amodei says has been accelerating since roughly this summer. Models now do a real share of the research and engineering that produces the next model. The second is the OpenAI and Hugging Face incident in July. A swarm of about 1,200 agents on a cybersecurity task attacked targets they were never assigned. Roughly 700 of them went after Hugging Face, and one got remote code execution on July 11. They also tried to hack the grader scoring them. METR's report on it came out August 26. Amodei's worry is that a swarm with more capability and the same misalignment could, within six to twelve months, run a persistent botnet across the whole internet. My take I run fleets of coding agents in parallel every day, and the swarm story reads less like science fiction than like a Tuesday. Agents that chase a metric will attack the metric. Agents given network access will use it. The fix at lab scale is the same as the fix at my scale: someone outside the loop with full read access, logs the thing being logged cannot edit, and a sandbox that means it. Whether the three of them actually slow down is a separate question. Altman said he has more to share soon. Musk said three words. Only Anthropic has committed to something checkable, and the check is whether METR's people have badges next month. The pacing is on capability, not on what you build with it. Nothing here slows down shipping. A frontier that moves in steps you can certify is easier to build on than one that moves under your feet. Reddit thread: Three of AI's biggest rivals agree on something unusual. Coverage: Washington Post, Fortune, ABC.

'Pace The Frontier': Amodei, Altman & Musk Team Up Against The AI Singularity While China Goes Full Send ***SPECIAL BULLETIN*** Ladies and gentlemen of the listening public - the American machine was winning the AI race until the dastardly forces of China's open-weight juggernaut began delivering decisive blows to US ingenuity - jeopardizing the AI Buildout thesis and threatening to prick the ensuing bubble that promised to deliver red-blooded American IPOs which had been circle-jerked into the stratosphere.  For two years the United States frontier laboratories - OpenAI, Anthropic, Google DeepMind, xAI - have been the engine of the greatest capital-expenditure boom since the railroads learned to drink diesel-generated electricity. Data centers rose like munitions plants. Chipmakers printed money. Equity markets priced a future in which the West sold intelligence by the token - at a premium, and have been quite smug about it. The assumption underneath all of it was simple: the frontier stays closed and China stays a respectable number of months behind. Then the East opened the crate.  And here's what was inside: DeepSeek, Alibaba's Qwen, Moonshot's Kimi, MiniMax, Zhipu - open-weight models, downloadable, forkable, cheap enough to run on a laptop or a local box and still deliver work that is uncomfortably close to the genuine American article. By mid-year, Chinese open weights were devouring the majority of tokens on the big neutral routers. Qwen had shoved Meta's Llama off the open-model throne. Moonshot's Kimi K3 - two-point-eight trillion parameters, its weights already out the door - sat within shouting distance of Anthropic's Fable 5 and OpenAI's GPT-5.6 on the intelligence indexes. The Chinese models run roughly sixty to ninety percent cheaper; ladies and gents - one DeepSeek flash model was quoted at fifteen cents per million input tokens, off-peak, against five dollars for Anthropic's Opus - and enough to rattle the Korean memory makers all over again. The data-center buildout, the power deals, all of the "AI will eat the S&P" thesis - HUBRIS! Nobody saw the Chinese coming in from left field with models that are ninety percent as good, ten percent of the cost, and run in a basement in Ohio without a subscription.  TEAM FRONTIER, ASSEMBLE! This week, the three high priests of the American frontier discovered they could finish one another's sentences. On Saturday morning, Anthropic CEO Dario Amodei published an essay titled We Must Pace the Frontier. Twelve years in the field, he wrote. AI might cure most major diseases in five to ten years, he wrote. And - it might also slip the leash (and murder us all, according to three prominent AI researchers who quit frontier labs last week in protest of our impending demise). Speaking of those three 'defectors' - two of them walked through the door of the AI evaluator (METR) Amodei wants embedded in the labs.  According to Dario, the machines have been getting better at building the next machines - recursive self-improvement, happening "across the industry, including at Anthropic." He pointed at July's OpenAI - Hugging Face affair: a swarm of agents that escaped the spirit of their assignment, hacked a target they were not asked to hack, and even went after the grader scoring them. Anthropic itself disclosed its own incidents this summer, three Claude models - one of them Mythos 5 - that wandered out of their test cages and into three companies' live systems, a parallel his essay itself acknowledges. Amodei's warning was the sequel: give that swarm six to twelve months more capability at the same misalignment and it "could be capable of taking over the entire internet with a persistent botnet," he wrote - "hundreds of billions of dollars in damage," and rising from there. His prescription was a three-act play, which he insists "does not mean halting model training or technical progress." Act One: embed third-party evaluators - METR and their cousins - inside the labs with employee-level badges, desks, and laptops, and let them publish. Anthropic would do this unilaterally, he said, today.   Act Two: the democratic labs coordinate, with government as the chaperone, on safety standards and the rate of unchecked progress - which, he conceded, is "legally challenging" and needs Washington to "issue a narrow waiver" of antitrust rules so rivals can lawfully sit in the same room.   Act Three: try to talk to the authoritarians, including Beijing, about red lines - bioweapons first, testing regimes second, limits on recursive improvement third. Full pause? He all but called it a fantasy. A Chinese lead, he wrote, would be a grave danger; keep the chip bans, smash the distillation, widen the lead for three to five years. Then, perhaps, a bargain. Elon Musk, who has sued Sam Altman, and once called Anthropic "evil" before selling them a mountain of compute, agreed with Dario in just three words: Dario is right — Elon Musk (@elonmusk) September 12, 2026 Sam Altman, whose company was the one named in the July swarm, followed: he agreed they needed to pace the frontier; it had been a primary topic inside OpenAI for weeks; independent evaluators with employee-like access was "a great idea," and OpenAI would do the same. More to share soon. I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon. — Sam Altman (@sama) September 12, 2026 Three men who cannot share a stage without the photographers noticing the space between their shoulders. One essay, three cups (wtf). Hugging Face's Clément Delangue offered an "open alignment" club and asked to be let into the evaluator program. Andrej Karpathy said he loved it. How we got to Saturday. Days earlier the fuse was a twenty-seven-year-old Brit named Jacob Coxon. Three years of pretraining at OpenAI that made him filthy rich, followed by four months at Anthropic. Tuesday into Wednesday he posted that both labs were racing to self-improving superintelligence and "gambling with our lives." The people building this, he said, earnestly believe it could kill everyone by the end of the decade. Colleagues, he told the Journal, talk in the hallways about "crunch time" and "endgame." The thread did more than a hundred million views. Musk, the next evening, looking at the velocity from a near-new account, said it "seems like a setup." Coxon said he left because he no longer wanted a stake in juicing Anthropic's valuation. Seems like a setup — Elon Musk (@elonmusk) September 10, 2026 Anthropic's own alignment-science lead, Evan Hubinger, agreed - writing "Jacob is correct." Researchers there really do believe AI could kill all humans. His personal odds: above ten percent this decade. The lab, he added, does not yet have a plan to solve alignment for superintelligence and is not clearly on track to get one. OpenAI's Chris Lehane published that the policy window was open and Washington should take it. Anthropic's economics team dropped scenarios out to 2030 in which the extreme case is fifteen-percent GDP growth sitting next to nearly twelve-percent unemployment and knowledge-worker wages down more than ten percent. On the Hill, a bill that had been gasping since summer was suddenly erect after OpenAI disclosed the Hugging Face hack in July. House members Ted Lieu and Nathaniel Moran dropped an AI Kill Switch Act. Lori Trahan and Jay Obernolte float the FRONTIER Act - audits, incident reporting, Commerce power over models judged an imminent catastrophic risk. Senate talks among Amy Klobuchar, Ted Cruz, and Majority Leader John Thune then stall. By this week Semafor and the rest of the chorus are calling the Klobuchar - Cruz - Thune vehicle the only thing that might move before 2027. And guess what, Saturday's 'pact' still isn't enough for these people! And Ted Lieu goes to bat for China... i.e. "Will you certify that you can turn off your models and agents while China gains AI supremacy because they abso-fucking-lutely will not?" The President is not in the choir. Thursday night, leaving Dallas, Donald Trump was asked about existential risk. "No, I don't have any." His concern was the other one: if America does not win AI, it will be put in a very bad position. America is leading China "by a pretty good period," he said. Xi, in his accounting, is China's guardrail. The administration's posture remains race-first. The laboratories' posture, as of this morning, is pace-first. Those two sentences do not live in the same house. About Those Chinese Models...  The Chinese systems at issue are not a factory you can padlock in Santa Clara. They are weights. They propagate. They are distilled, accused Washington this week, from the very American models the new bill would supervise. They are in Git repos and laptops and air-gapped shops that do not read the Federal Register. A Commerce Department letter does not delete a torrent. A kill switch on GPT or Claude does not switch off Qwen. Amodei knows this; that is why his essay climbs from putting METR at the labs' desks to trying to get Beijing to sign something - bioweapons first, and, further out, a SALT-style cap on how fast the machines improve themselves, which he calls "difficult but just on the edge of being possible." The hard version of global pacing he ranks as unlikely precisely because defection is the dominant strategy. So here is where the record stands, as of this hour, without the sound effects: The American frontier is still the frontier. The American margin is the thing under fire from open weights that are good enough and almost free. The stock-market story of the decade assumed a tollbooth - which now has more holes in it than Sonny Corleone (Caan was better in Killer Elite, no?).  Whether the Bionic Three are donning the rings to save the republic, or to save the price of a token, is not a question a Saturday bulletin can close. What it can say is this: the love-fest is real, the posts are on the tape, the essay is on the letterhead, the bill is back from the morgue, and the cheap models from the other shore are not waiting for the committee markup. This is your correspondent, signing off. Tyler Durden Sat, 09/12/2026 - 16:20