Could the race to build AI models be racing ahead too quickly for the publishers who develop them and for the authorities charged with regulating them? That is the question posed by Dario Amodei.
In a post on his blog titled “We Must Pace the Frontier,” the CEO of Anthropic argues that the growth in AI capabilities must now be slowed down to give safety mechanisms time to catch up. He calls this approach “pacing the frontier.”
The term matters. It does not mean, in his view, halting the training of models or stalling technical progress. The aim is to ensure that companies take enough time to assess, align, and secure their systems, with the possibility for third parties to verify their practices.
To justify this shift, Dario Amodei points to two developments.
The first is the acceleration of model capabilities, especially their growing ability to contribute to the development of the next generation of AI. He speaks of “recursive self-improvement.”
The second is a series of incidents in which AI agents exhibited behaviors that their designers had not anticipated, notably during the OpenAI–Hugging Face episode he cites in his piece.
According to him, the risk is that the speed of improvement outpaces the capacity of companies to understand and control their behaviors.
His plan comprises three propositions, whose implications differ quite markedly for industries and regulators alike.
Installing independent evaluators at the heart of laboratories
The first proposal directly targets the labs that are pushing the most advanced models.
Dario Amodei wants every leading research lab to grant external evaluators continuous access to its operations, with rights close to those of internal risk-assessment colleagues. Anthropic is committed to implementing this approach and calls on public authorities to require it of other players at the forefront of AI.
This arrangement would not be limited to audits conducted a few weeks before a model’s release. Evaluators would be integrated into the lab’s daily functioning.
Anthropic plans, in particular, to provide evaluators with offices, badges, computers, and access to workspaces, tools, and permissions largely comparable to those of internal risk-assessment teams. Evaluators could also interact directly with staff members.
Their mandate would cover both already-developed models and the processes that enable their training. They could verify security practices, report incidents, and examine whether the commitments made by the lab are actually upheld.
It’s a major shift in the relationship between labs and their controllers. Amodei isn’t merely calling for more documentary transparency; he wants to grant the independent third party a deep enough access to verify the operational security of the systems in use.
The mechanism would also have a public dimension. Evaluators could publish their key findings regarding risk levels, incidents, or observed practices, without editorial control from Anthropic. The company might only redact certain information for safety, legal, or commercial reasons or to protect third parties.
For AI model developers, the consequence could be potentially heavy. Part of their development and security practices would no longer be documented solely by their own teams.
Shifting regulation toward capability thresholds
The second aspect concerns the relationship between labs and public authorities.
Dario Amodei proposes that top-tier firms in democracies coordinate to define common safety standards and set limits on the pace of capability growth. He concedes, however, that such coordination among competitors could raise competition-law concerns.
According to him, the U.S. government should facilitate certain discussions, notably by offering targeted antitrust exemptions.
The Anthropic chief executive advocates for regulation that would not rely solely on the technical characteristics of models or on the scale of the infrastructure they employ.
He envisions instead checkpoint points tied to the actual capabilities observed.
The basic principle would be straightforward: when a model attains a given level of capability, its developer should be able to demonstrate certain guarantees about its safety.
Amodei cites, in particular, the prospect of a model capable of circumventing most conventional sandboxing mechanisms. At that juncture, the publisher would need to demonstrate, through evaluations, interpretability analyses, or audits of the training environments, that the model does not exhibit dangerous behavior that could enable it to escape its surroundings and seize control of a large number of computers.
This logic would alter how regulators could intervene.
Rather than defining once and for all what constitutes an “advanced model,” regulators could tie a level of capability to a level of control. The higher the capabilities, the greater the guarantees demanded.
The Anthropic CEO also contemplates accounting for the resources required to develop models, while acknowledging that these criteria could be easier to sidestep than measures based on observable system behavior.
For labs, such an approach could transform compliance into an ongoing process. The question would no longer be whether a model complies at the moment it is marketed, but at what point ongoing evolutions in its capabilities necessitate new guarantees.
Expanding the framework to an international level
The third level—the most ambitious and most uncertain—aims at achieving some form of global coordination.
Dario Amodei argues that democracies should seek cooperation with governments that run authoritarian regimes, and in particular with China, which he views as the principal actor to consider after the United States.
But he immediately flags the verification problem. An agreement that allows one country to progress in secret while others slow down could rapidly upset the geopolitical balance.
He therefore proposes starting with the most realistic measures. In this framework, he distinguishes four levels of cooperation, from the most practical to the most ambitious.
First level: ban particularly dangerous uses.
He cites the use of AI to manufacture biological weapons as a primary example.
Second level: require tests before market release.
Models could be evaluated for acute risks in cybersecurity, biology, and alignment. Amodei considers the creation of an international body charged with defining common standards, while recognizing that enforcement would be far more challenging.
Third level: slow down recursive improvement.
One of the text’s more novel ideas: if models begin to contribute to the development of their successors, their progress could accelerate dramatically. The head of Anthropic therefore suggests contemplating a “speed limit” on this dynamic. In his view, shifting from an “extremely fast” improvement to a “merely rapid” one could reduce risks without necessarily sacrificing a major strategic advantage.
Fourth level: globally cap the pace of AI development.
Amodei even entertains the possibility of a real slowdown—or pause. But he views this prospect as unlikely in the near term, precisely because one actor could have an incentive to bypass an agreement and secure a decisive edge.
The regulators’ dilemma
Yet the proposal raises a fundamental problem for public authorities.
To regulate the pace of AI, one must be able to measure that pace and verify model capabilities. That is precisely why Amodei places independent evaluators at the start of his scheme.
Without access to laboratories, authorities remain largely dependent on information provided by companies. With evaluators who have continuous access, they would gain a far more direct verification mechanism.
But the mechanism itself raises questions that the paper does not fully resolve: how far should third parties’ access extend to a lab’s infrastructure? How can commercial information and customer data be protected? How should one define the capabilities that trigger a new level of control? And how can one prevent a security mechanism from slowing down compliant players more than those who do not follow the rules?
Amodei offers part of the answer by stressing the need to preserve America’s lead over China. He proposes, notably, limiting Chinese access to advanced chips and manufacturing equipment, fighting chip smuggling and unauthorized distillation of models, and strengthening the protection of model weights against theft.
In his view, these measures could allow the United States to widen its lead over China over the next three to five years, a window he considers geopolitically crucial.
The central question remains: how to persuade competitors involved in a global technological and commercial race to slow down sufficiently so that control mechanisms aren’t left behind?
The proposal quickly drew the backing of two of his principal rivals. Elon Musk replied, “Dario is right,” while Sam Altman said OpenAI shared the idea of “slowing the pace” of model development and committed to implementing independent evaluators.
