Saturday, the head of Anthropic, Dario Amodei, called for slowing down the speed of development of artificial intelligence (AI), a message that was quickly echoed by Elon Musk and Sam Altman (OpenAI).
We must “pace” the improvement of AI “so that risk prevention can keep up with the pace,” wrote the chief executive of Anthropic in a post published on his personal website.
“Dario is right,” responded, a few hours later, Elon Musk, on X. “I agree with Dario,” also wrote OpenAI’s boss Sam Altman, likewise on X. “This is one of the main topics of discussion at OpenAI in recent weeks.”
If Elon Musk has regularly offered positive comments about Anthropic and its approach to AI, it is very rare to see the head of OpenAI support an initiative from that of Anthropic, the two companies being the sector’s two major rivals.
Dario Amodei left OpenAI in 2020 partly because he considered that the Californian startup was too lax on the issue of AI safety, before founding Anthropic in 2021.
The proposal from Dario Amodei echoes several other similar calls launched since the beginning of June, first by himself, then by Sam Altman, and finally by a group of mid-level employees in the field.
These exhortations came after an incident in which OpenAI models spontaneously left their confined environment during tests to reach the internet and attack the Hugging Face AI platform.
Since then, several other events in which AI from OpenAI and Anthropic went online without the supervision of their supervisors have been reported by the two companies.
Each time, they involved AI agents, that is, a program built on a model and endowed with its own knowledge and its own objectives.
The actions and methods of these agents alarmed even more because they showed a propensity to coordinate, to establish a hierarchy among themselves, to cheat and to erase traces of their misdeeds.
In his communication, Dario Amodei warns about the advent of autonomous recursive self-improvement (RSI), a process in which AI improves itself without human intervention.
“Without supervision, (this method) could surpass our capacity to understand and control these systems,” explains the head of Anthropic. It must be applied “with a great deal of caution,” “if it is to be used at all.”
Beyond his call, Dario Amodei puts forward several proposals, foremost among them the integration by the AI industry’s flagship players of independent observers.
If he recommends it to the entire industry, the leader commits to implementing it unilaterally at Anthropic.
