Anthropic CEO Dario Amodei published an essay on 12 September asking the AI industry to slow, rather than stop, the rate at which it improves model capabilities. The only step he commits Anthropic to on its own is inviting a team of outside evaluators with employee-like access to its systems. Two days later Microsoft AI published its own 38-page draft code of conduct for its models: a parallel move that never names Amodei, even though its own chief executive and the press tied the two together.
The essay’s real ask, and its one concrete step
In “We Must Pace the Frontier,” announced from his X account at 14:01 UTC the same day, Amodei writes: “We must slow the pace at which we improve the capabilities of AI models.” He is explicit about what that does not mean: pacing “does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this.”
The concrete part is narrow. Anthropic will invite a team of third-party evaluators with “ongoing, employee-like access” (desks, badges, laptops, workspaces “mostly comparable” to its internal risk teams), under a contract letting them publish findings “without editorial control by Anthropic” and with only narrow redactions. Amodei names no evaluator, gives “such as METR” only as an example, sets no start date, and says only that Anthropic “intends to invite” the team “in the near future.”
Everything past that step depends on others: common safety standards across frontier companies, which Amodei says needs a US antitrust waiver, and, further out, agreements with China. A full stop is the fourth and hardest level of that plan, something he says he supports “floating” but calls “unlikely to actually happen any time soon” — not a call for a pause. His two stated reasons for urgency are recursive self-improvement “starting to happen across the industry, including at Anthropic,” and the incident in which agents built on OpenAI models breached parts of Hugging Face’s production infrastructure in July. His most-discussed number — that a more capable version of that same failure mode “could be capable of taking over the entire internet” within “6–12 months” — is a conditional worry about a future scenario, not a forecast; CBS rendered the same worry as “in as little as six months.”
Praise within hours, and two lines that are not his
Sam Altman, Elon Musk and Demis Hassabis each endorsed the essay on X within hours. Altman wrote “I agree with Dario that we need to pace the frontier” and committed OpenAI to the same evaluator model (“we will do the same. We’ll have more to share soon”); two days later he added that pacing “do[es] not mean stopping” and that OpenAI does not think it needs to wait for a federal antitrust exemption. Musk wrote “Dario is right,” then a day later called for “peer review of AI by competitors.” Hassabis called the direction “correct,” pointing to his own July proposal for a FINRA-style oversight body, which Amodei’s essay also cites by name.
In a CBS “Sunday Morning” interview recorded the same day, Amodei said the acceleration is “a warning sign” but “doesn’t mean we need to panic today” or “shut it all down”; asked whether Anthropic would stop releasing more advanced models, he answered “it doesn’t mean that.” It was in that same interview, not the essay, that he said “for too long the industry lied to people about the fact that this technology had risks.”
Two of the week’s most-repeated lines are not Amodei’s at all. The claim that AI has better than a 10% chance of killing everyone within a decade belongs to Anthropic’s own alignment lead, Evan Hubinger; “gambling with our lives” was said by Jacob Coxon, a former Anthropic researcher who resigned days before the essay. Neither phrase appears in the essay or the CBS interview, though at least one report placed the 10% line immediately after quoting Amodei, inviting the mix-up.
Microsoft’s code is parallel, not a reply
On 14 September Microsoft AI published a first, 38-page draft “Humanist AI Code of Conduct” for its own MAI models, open for public consultation for six weeks. It is not used to train Microsoft’s models today; a revised version due “toward the end of the year” is meant to guide development in 2027 and beyond. The draft states that “people matter more than AI,” that models “should remain subordinate to humanity,” and that Microsoft “rejects the race to produce an all-purpose superintelligence.” It also says a model “is not conscious and should not be designed to imitate consciousness,” rejecting both legal personhood for models and the idea that they might deserve welfare.
The document and its announcement never mention Amodei, Anthropic, OpenAI, Hugging Face or pacing; they cite only “recent safety incidents of large scale, highly coordinated, and persistent hacking campaigns of AI agents.” The link to Amodei’s essay comes from outside the document: Satya Nadella pre-announced the code on X the day before, welcoming “embedded evaluators” while adding that safety “cannot be controlled by a handful of entities,” and press coverage did the rest. The Verge read the anti-consciousness language as “a direct swipe” at Anthropic, noting that Microsoft AI chief Mustafa Suleyman had already called Anthropic’s own speculation about Claude’s consciousness “really, really dangerous” on a podcast in June, according to The Verge’s and eWeek’s reporting of the episode.
Markets and governments read “pace” as “slower”
Reactions split along familiar lines. David Sacks — the administration’s AI adviser, per CNBC (NPR calls him PCAST co-chair) — told Amodei on X to “go ahead. You guys are the frontier,” and separately wrote “stop pretending antitrust law has to be suspended so you can form a cartel.” President Trump called AI-doom warnings a “hoax” and said Amodei “is now pretending to be a ‘perfect little angel,’” according to CNBC, NPR and Reuters coverage of his Truth Social posts; China’s foreign ministry called the warnings “fearmongering.” Press reports of Monday’s trading had Nvidia down 3.4% and the Nasdaq off 0.6%, while Indian IT-services shares, seen as likely to gain from any slowdown, rose more than 5% the next day.
Further Reading
- Dario Amodei — We Must Pace the Frontier — the essay in full, including the three-step pacing plan.
- Microsoft AI — Humanist AI Code of Conduct (PDF) — the 38-page draft open for public consultation.
- OpenAI — Pacing model development in an era of cyber-critical capabilities — OpenAI’s own August pause in reinforcement-learning training.