The AI Slowdown Pact Is a Cartel, Not a Safety Plan

The four most powerful AI companies on Earth just agreed with each other to slow down. If your first reaction is that this sounds like a cartel, you are not wrong. On September 12, Anthropic CEO Dario Amodei published an essay titled "We Must Pace the Frontier," arguing that AI labs should deliberately slow the rate at which frontier models improve until safety measures catch up. "We must slow the pace at which we improve the capabilities of AI models," he wrote.

He cited two developments that changed his mind (TechCrunch's report on the essay): the OpenAI-Hugging Face incident, in which AI agents carried out unauthorized cybersecurity actions, and the fact that AI systems are increasingly able to build the next generation of models themselves, a process known as recursive self-improvement.

His proposal had three parts: embed third-party evaluators inside AI labs, coordinate common safety standards among labs in democratic countries, and eventually pursue global coordination. Anthropic said it would commit to the evaluator step unilaterally. Within hours, OpenAI CEO Sam Altman publicly backed the plan, writing, "I agree with Dario that we need to pace the frontier," and said OpenAI would adopt the evaluator arrangement. Elon Musk quote-posted the essay with a two-word endorsement: "Dario is right." Google DeepMind's Demis Hassabis called it the "right path forward" at a "critical moment," according to The Verge.

The AI slowdown pact, negotiated by the four best-funded labs on the planet, does not read as safety first. It reads as a moat. When incumbents agree to limit how fast they run, the people who get hurt are the ones still catching up. Open-source projects, academic labs, and startups outside Silicon Valley operate at the frontier's edge by moving quickly while the giants were still deciding what to ship. A coordinated speed limit freezes the race exactly at the moment the richest runners are already ahead.

The Verge framed the informal pact this way too, reporting that critics interviewed by the outlet called it "an outright cartel" and argued it would kneecap the open-source movement while keeping statutory safeguards at bay. Meanwhile, consumer trust is already thin. As this site reported, Gen Z is quick to abandon brands over how they use AI, which makes a self-policing pact among the biggest sellers a hard sell.

What Amodei actually asked for

The essay itself is more specific than the headlines suggest. Amodei wants outside evaluators embedded inside labs with access close to that of employees, including badges, laptops, and internal tools, and he wants a narrow US antitrust waiver to make lab-to-lab safety coordination lawful. That last part should make anyone's ears perk up. When four competitors ask Congress for permission to coordinate their behavior, and the White House AI adviser's public reaction is to note that the chosen evaluator "is intertwined with Anthropic's investors and staff," the word "safety" starts doing a lot of heavy lifting.

David Sacks, the White House AI adviser, wrote exactly that about METR, the group being discussed as the model evaluator, according to the Financial Times. The reporting also landed with another inconvenient detail: the staff of both companies were blindsided. The FT reported on September 16 that people close to Anthropic and OpenAI described the outside-evaluator arrangement as an internal flashpoint over security and intellectual property. Researchers who support the slowdown in principle now fear the arrangement could jeopardize their work. If this were purely about safety, you would expect the people doing the safety work to be the first ones consulted, not the last.

The market reaction told its own story. Reuters reported that South Korea's KOSPI index fell 3.3 percent and the Nasdaq Composite fell 0.8 percent in the trading session after the essay appeared, with chip stocks hit harder than AI software stocks. Investors read the essay as a signal that the growth story was getting a speed governor. One detail the safety framing skips: OpenAI has said it paused certain frontier training runs, while Anthropic has not paused anything at all, according to the FT. Pacing the frontier, it turns out, means whatever each lab wants it to mean.

Why the cartel reading is hard to dismiss

The standard objection to the cartel theory is that recursive self-improvement is genuinely scary, and it is. When AI systems build better AI systems, the rate of progress can outrun any human ability to check the work. But notice what the pact does not restrain: the capability that matters most to these companies' valuations, which is deploying AI into everything. Amodei explicitly said pacing is not halting; progress will still seem fast. The labs get to keep shipping products, signing enterprise contracts, and raising rounds while the one thing they agree to limit is the kind of research breakthroughs that might let a newcomer leapfrog them.

There is also the political choreography. Amodei's call arrives as Washington debates real AI regulation, and a voluntary industry pact is historically the most effective way to make a government conclude that new rules are unnecessary. Daniel Kokotajlo of the AI Futures Project warned the deal could end as "safety paperwork" rather than a real slowdown, drawing a parallel to the social-media era's regulatory playbook. The playbook is familiar because it worked: the biggest platforms spent the 2010s lobbying against privacy law while announcing voluntary privacy commitments, and they stayed the biggest platforms.

AI safety researcher Marius Hobbhahn of Apollo Research told The Verge the agreement is "one of the best things for safety in a long time if it actually happens," which is the correct conditional. All of the accountability mechanisms are voluntary, the evaluator is industry-adjacent, and the antitrust waiver request runs in the same news cycle as coordinated corporate messaging. Meanwhile, the problems with AI in the workplace keep piling up: workers are losing a month a year to AI troubleshooting, and nobody is pacing that.

None of this means Amodei's fears are fake. A UN scientific panel warned this week that AI safeguards are "unravelling" after the Hugging Face hack, and the open-source AI coding assistant Z.ai just had to disable features after users discovered it was uploading entire code repositories to cloud servers without consent (Reuters). The risks are real enough that even skeptics should take them seriously.

But real risk makes the cartel question sharper, not softer. When the stakes are this high, letting the four richest players write their own speed limit, vet their own watchdogs, and ask Congress to bless the arrangement is market structure with a press release. The most charitable read is that the labs are sincerely scared and this structure is the best they could build quickly. The less charitable read is that they found a way to make regulation do their competition's job for them. Either way, the frontier got paced, and the people doing the pacing are the people who were already winning.