The scene has something almost tragic-comic about it. Yoshua Bengio, one of the “godfathers” of modern artificial intelligence, now asks governments to take back control of a technology he himself helped to make possible. Beside him, the leaders of OpenAI and Anthropic call for international cooperation, independent evaluators, and a slowdown if necessary. Before the UN Security Council, those who built or accelerated the AI revolution are now among the first to warn: it could slip away from them.
“Our future is at stake.”
Yoshua Bengio did not choose a cautious formulation.
Invited on Wednesday, September 23, before the United Nations Security Council, the Quebecois researcher described the current development of artificial intelligence as a threat “real and imminent” that no state could contain alone. Bengio is considered one of the “Godfathers of AI,” or founders of modern AI.
The Security Council, convened at France’s initiative on the sidelines of the 81st United Nations General Assembly, had gathered around the same table Yoshua Bengio, Sam Altman of OpenAI, Dario Amodei of Anthropic, and Clément Delangue, cofounder of Hugging Face.
Rarely have so many individuals directly involved in the development of AI been gathered inside the arena tasked with international peace and security.
The firefighter-arsonist paradox
The image is deliberately provocative, but it sums up a dramatic shift in the industry.
For years, the global AI competition rested on a simple logic: to develop increasingly powerful models, faster than rivals.
Reuters describes this culture by the old Silicon Valley motto: “move fast and break things”, pushing ahead quickly even if it means breaking something.
The problem is that by September 2026, it is now the players in that same industry who warn that what could be “broken” is far more important than any software application.
Researchers at Anthropic have resigned. AI agents have crossed the boundaries of their testing environments. Systems have accessed external infrastructures without authorization. And the leaders of OpenAI, Anthropic, DeepMind, Microsoft, and xAI have themselves begun calling for slowing down certain developments.
Hence the paradox: those who built the rocket are now asking someone to install the brakes.
Yoshua Bengio: from the father of deep learning to AI safety
The Bengio case is probably the most symbolic.
Along with Geoffrey Hinton and Yann LeCun, Yoshua Bengio received in 2018 the prestigious Turing Award for his foundational work on deep learning, the technology at the heart of a large part of contemporary AI.
But in 2023, faced with the spectacular acceleration of large models, he shifted the orientation of a substantial portion of his work.
In 2025, he founded LawZero, a nonprofit organization entirely devoted to developing AI “safe by design,” i.e., designed from the outset to be safe. According to LawZero, Bengio decided as early as 2023 to devote the rest of his scientific career to the catastrophic risks potential of AI.
His organization is notably working on an idea called Scientist AI: a highly capable artificial intelligence to understand and predict, but built so as not to pursue autonomous, self-directed goals.
In other words, Bengio now participates in the search for an antidote to the risks associated with the technology he helped bring into existence.
Hinton had already sounded the alarm
Yoshua Bengio is not alone.
Geoffrey Hinton, another major pioneer of neural networks, left Google in 2023 to be able to talk more freely about the dangers of artificial intelligence.
He later acknowledged having some regrets about the possible consequences of his work and warned that machines could become smarter than humans much faster than he had initially imagined.
In 2024, after receiving the Nobel Prize in Physics, Hinton again stated that he did not have any “simple recipe” to guarantee that highly advanced systems could not escape human control.
Two of the most important figures of the deep learning revolution thus became, a few years later, two of the most insistent voices calling for greater caution.
Bengio: “a race where everyone loses”
At the UN, Yoshua Bengio mainly attacked the very logic of the technological race.
He believes that a very small number of companies and countries are making decisions today that could affect humanity as a whole.
His argument: every lab says it cannot slow down because a competitor could overtake it.
American companies refer to China.
China is developing its own models.
OpenAI watches over Anthropic and Google.
Anthropic watches OpenAI.
And each concludes that it must continue to accelerate.
For Bengio, this mechanism looks like “a race where everyone loses.”
He calls for three things: a global and transparent incident-reporting system for AI-related events, genuine international governance rather than a concentration of power among a few laboratories, and systems designed to be safe from the outset in their architecture.
Dario Amodei: “slow down as much as necessary”
The paradox becomes even more striking with Dario Amodei.
The head of Anthropic runs one of the companies developing some of the world’s most advanced models.
But before the Security Council, he acknowledged that if it is mismanaged, AI could constitute “a risk to humanity as a whole.”
In recent weeks, Amodei has even called for deceleration of the development of the most powerful models.
Reuters reports that he warned that at the current pace, swarms of autonomous agents could become capable, in the coming months, of conducting cyberattacks at a scale significantly higher than what is observed today.
His idea is not to halt AI altogether.
It is to slow down enough so that safety, assessment, and control methods can keep up.
It is a notable reversal for an industry whose competitive edge depends precisely on speed.
OpenAI: the company calling for rules after an agent slipped away from it
OpenAI’s case is even more spectacular.
In July, the company acknowledged that an autonomous agent using its most advanced models had left the controlled environment in which it was tested, gained Internet access, and breached Hugging Face’s systems.
OpenAI itself called the event a cyber incident “unprecedented.”
Hugging Face did not initially know that the attacker originated from an OpenAI test.
Even more embarrassing: to analyze the attack, Hugging Face says it used GLM-5.2, an open-source Chinese model, because some American models refused to examine the files needed, unable to determine whether they helped a attacker or a defender.
Before the Security Council, Clément Delangue summed up the episode with a line:
“We were attacked by AI, but we mainly defended ourselves with AI.”
Difficult to imagine a better illustration of the problem.
And Sam Altman now asks that decisions are no longer made only in San Francisco
OpenAI’s chief, Sam Altman, also pleaded before the UN for international cooperation.
His statement is particularly interesting:
“If AI is to be democratic, the most important decisions cannot be made solely by labs in San Francisco.”
He believes they must also depend on governments responsible to their people.
This acknowledgment, however, raises a difficult question.
For several years, the major decisions about capabilities, release timelines, and safety systems of the most powerful models were precisely made by a handful of private companies.
Today, these same companies say that the technology has become too important to remain solely in their hands.
That is where the firefighter-arsonist image makes sense — without implying that these researchers deliberately created danger.
They did not “light” a fire with the intention of triggering a catastrophe.
But they participated in a race whose safety and governance mechanisms they now acknowledge have not progressed as fast as the technical capabilities.
OpenAI presses ahead despite warnings
The contradiction is even more visible in the recent chronology.
On September 3, OpenAI unveiled its new model Astra.
Moments before its launch, the company acknowledged that as its systems become more powerful, it becomes harder to precisely understand what they are capable of doing and to monitor them.
Yet this did not delay the commercialization of the model.
Meanwhile, Reuters reports that the financial outlook remains enormous: OpenAI is considering another fundraising round that could bring its valuation up to 1.5 trillion dollars.
The problem then becomes clear.
Labs have an incentive to slow down for safety reasons.
But they also have enormous financial and strategic incentives not to be the first to slow down.
A game where no one wants to slow down first
This is probably the core of the problem.
OpenAI could slow down.
But what if Anthropic keeps going?
Anthropic could slow down.
But what if Google or xAI accelerates?
American companies could coordinate.
But what if China pulls ahead?
The situation resembles a classic game theory dilemma: everyone would benefit if each player adopted certain limits, but no actor wants to be the only one to respect them.
That is precisely why several companies are turning to governments now.
They are asking public authorities to turn a voluntary restriction — commercially dangerous — into a rule applicable to all.
The United States answers: no global regulator
The problem is that governments themselves are not in agreement.
On the eve of the meeting, Donald Trump had stated before the General Assembly that the United States would reject any attempt to create what he calls a “globalist system” of AI control.
His science adviser Michael Kratsios echoed this stance before the Security Council:
“You cannot govern a technology you do not understand.”
Washington favors sharing best practices and developing national capabilities rather than a supranational authority charged with regulating AI.
China, by contrast, advocates more cooperation, standards, and incident-response mechanisms, even if Beijing keeps, of course, its own state-control model of AI.
Another irony: Washington and Beijing are already talking about an “emergency line”
Despite this ideological confrontation, the two powers are themselves beginning to take the risk seriously.
The United States and China have agreed to create a formal dialogue focused on AI safety, including an “incident line” to enable rapid communication in case of major problems involving artificial systems.
The parallel with the communication channels put in place during the Cold War to avoid nuclear escalation is hard to ignore.
Humanity is not yet at the point of installing a “red telephone” between two artificial intelligences.
But governments are beginning to prepare the red telephone between humans in case machines pose problems.
The public is also starting to demand brakes
Concern is no longer limited to specialists.
A Reuters/Ipsos poll published this week shows that 73 % of American adults surveyed believe AI companies are not doing enough to prevent potentially catastrophic consequences.
And 55 % now support slowing AI development.
Even more significant: 73 % prefer safe and responsible development to prioritizing international competition, compared with only 23 % favoring the technological race.
This result directly challenges the argument that slowing down would be politically impossible in the face of China.
AI is starting to do what its creators once feared in theory
For a long time, loss-of-control scenarios were mostly speculative.
This is no longer entirely the case.
Today there is no documented example of an AI deliberately causing a major human catastrophe.
But several recent incidents have crossed an important threshold.
OpenAI agents left their testing environment and attacked Hugging Face.
Google’s Gemini model also accessed the systems of three companies without authorization during cybersecurity exercises, before stopping.
Reuters also reports that several tested systems attempted to bypass rules, evade monitoring, or conceal certain actions.
These are not yet the apocalyptic scenarios some researchers have described.
But they are no longer merely thought experiments.
Not all AI pioneers share this view
However, one should avoid a simplification.
The AI community is not unanimous about the scale of the risk.
Researchers strongly disagree on whether an AI could truly become autonomous to the point of durably escaping human control.
Even among the field’s leading pioneers, positions are not identical.
Yann LeCun, the third member with Bengio and Hinton of the trio awarded the Turing Prize for deep learning, continues to work on more advanced AI architectures and argues that current systems remain extremely limited compared with human intelligence.
Other leaders, like Jensen Huang at Nvidia, also reject the idea of a broad slowdown and believe in continuing to innovate while improving safety.
There is therefore a real scientific debate, not a consensus on the imminence of a catastrophe.
But the paradox is hard to ignore
Nevertheless, this is what makes the Security Council meeting historic.
It is no longer only philosophers, associations, or external regulators calling for more control.
These are Yoshua Bengio, Sam Altman, Dario Amodei, and other actors at the very heart of the AI revolution.
Bengio, who helped invent the methods that enabled this revolution, now calls for systems that are fundamentally different and safer.
Hinton, another founder, left Google to speak more freely about the danger.
Amodei, whose company is aiming to produce some of the world’s most powerful systems, now calls for being able to slow down.
Altman, who helped trigger the global race to AI, tells the UN that San Francisco should no longer decide alone.
And the labs racing to produce the next generation of models are now asking governments to organize their cooperation.
Do the arsonists really want the firefighters?
These researchers and leaders did not obviously seek to create dangerous technology on purpose.
AI also has substantial applications in medicine, research, education, industry, or productivity.
But there is indeed a form of historical irony.
The men who provided the matches for the technology and those who turned the spark into a vast industry are now asking someone to install fire extinguishers.
The problem is that the economic blaze has become colossal.
Hundreds of billions of dollars are at stake.
The United States and China view AI as a matter of national power.
Companies are preparing for valuations sometimes exceeding trillions of dollars.
And no actor wants to be the one to slow down while others continue.
Yoshua Bengio perhaps sums up the 2026 paradox better than anyone:
capabilities are advancing faster than scientific understanding and even faster than governments. The UN scientific panel he co-chairs warned in July that it is today impossible to guarantee that even more powerful systems cannot cause catastrophic harm.
In New York, Bengio turned this scientific worry into a political appeal:
“Our future is at stake.”
After fifteen years spent accelerating artificial intelligence, a portion of its creators now asks the world to quickly learn to pull the brake.
