Calls Grow to Halt development of Superintelligent AI,Signed by Nobel Laureates and Celebrities
A growing chorus of voices-including five Nobel laureates,tech pioneers,and prominent figures like Prince Harry and meghan Markle-is demanding a prohibition on the development of superintelligent AI until safety and societal alignment can be assured for humanity.
The letter, spearheaded by the Future of Life Institute (FLI), calls for a halt to development “not lifted before there is broad scientific consensus that it will be done safely and controllably, and strong public buy-in.” This marks a shift from the FLI’s 2023 call for a six-month pause on all powerful AI systems, focusing now specifically on the risks posed by AI exceeding human intelligence.
The signatories represent a remarkably diverse coalition. Among them are individuals described as the “Godfathers of AI,” Apple co-founder Steve Wozniak, and Steve Bannon, a political strategist. The inclusion of Paolo Benanti,an advisor to Pope Francis,and the Duke and Duchess of Sussex underscores the breadth of concern extending beyond the scientific and technological communities.
Defining Superintelligence and the Urgency of the Moment
The FLI defines superintelligence as an AI system capable of surpassing human performance across all useful tasks. Organizers believe this threshold could be reached within as little as one to two years, fueling a sense of urgency. “Time is running out,” stated Anthony Aguirre,the FLI’s executive director,in an interview. He believes the primary obstacle to unchecked development is a lack of widespread public awareness and concern.
Recent polling data supports this assertion. According to data released alongside the letter, 64% of Americans believe superintelligence should not be developed until it is demonstrably safe and controllable, while only 5% advocate for rapid development. “It’s a small number of very wealthy companies that are building these, and a very, very large number of people who would rather take a different path,” Aguirre explained.
A Broad range of Voices Express Concern
The letter has garnered support from a wide array of public figures. Actors Joseph Gordon-Levitt and Stephen Fry,rapper will.i.am, and author Yuval noah Harari have all added their names. Notably, Susan Rice, who served as national security advisor under President Barack Obama, is also a signatory. Even a member of staff at OpenAI-the company CEO Sam Altman has described as a “superintelligence research company”-Leo Gao, has publicly endorsed the call for caution. Aguirre anticipates further endorsements as the campaign gains momentum,suggesting that many share these concerns privately.
prince Harry, Duke of Sussex, articulated a broader philosophical point in a message accompanying his signature: “The future of AI should serve humanity, not replace it. I believe the true test of progress will be not how fast we move, but how wisely we steer. There is no second chance.”
Actor Joseph Gordon-Levitt’s accompanying message highlighted specific anxieties about the potential misuse of advanced AI, questioning whether it needs to “imitate humans, groom our kids, turn us all into slop junkies and make zillions of dollars serving ads?” He argued that this is the direction many tech companies are heading under the guise of building “Superintelligence.”
The Need for Regulation and a Shift in Incentives
While the letter’s intentionally minimal wording was designed to attract a broad base of support, organizers recognize that meaningful change will likely require government intervention. Aguirre emphasized the need for regulation to address the “perverse incentive structures” driving companies in the United States and China to compete for dominance in superintelligence development.
“Whether it’s soon or it takes a while, after we develop superintelligence, the machines are going to be in charge,” Aguirre warned. “Whether or not that goes well for humanity, we really don’t know. But that is not an experiment that we want to just run toward.”
