Anthropic safety researcher Evan Hubinger has estimated a greater than 10 percent probability that advanced artificial intelligence could cause human extinction this decade, sparking international security warnings and intersecting with Amazon’s massive capital investments in AI infrastructure.
The debate over artificial intelligence safety escalated when senior researchers publicly voiced extreme concerns regarding the future trajectory of autonomous intelligence. The discussion gained urgency following a staff departure at artificial intelligence developer Anthropic, which prompted sharp exchanges across public platforms about the risks of rapid development cycles and self-improving systems.
Internal Warnings and Safety Concerns at Anthropic
The public discourse intensified when researcher Jacob Coxon announced his departure from Anthropic in a post on the social platform X. Coxon accused leading AI laboratories of racing straight to self-improving superintelligence and gambling with human lives despite believing that the technology could ultimately prove fatal to humanity.
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.”
Jacob Coxon, former Anthropic researcher
Responding to the departure, Evan Hubinger, an alignment science lead at Anthropic, confirmed deep internal apprehensions regarding the capabilities of advanced models. Hubinger stated on X that we really do earnestly believe AI could kill all humans, putting the probability above 10 percent for the current decade while conceding that the organization lacked a clear alignment plan to control superintelligence.
The warnings align with outside assessments from prominent figures in the field. Artificial intelligence pioneer Geoffrey Hinton told interviewers that a 10% chance seems not an unreasonable estimate given that humanity has never created entities potentially smarter than itself. Hinton noted that malicious systems would not require physical control to cause devastation, suggesting they could generate complete chaos through persuasive communication and cyberattacks.
Political Fallout and Calls for Global Regulation
The researchers’ admissions quickly resonated inside government circles, triggering policy debates in the United Kingdom and prompting calls for multinational oversight. Prime Minister Burnham acknowledged during parliamentary questions that AI poses risks to our national security while simultaneously emphasizing its potential utility in public safety applications. To elevate internal governmental discussions, the administration granted AI safety minister Kanishka Narayan a seat at the Cabinet table.
Lawmakers pushed for more aggressive structural interventions. Former Cabinet Office minister Darren Jones wrote to international leaders, including the United Nations and the Organisation for Economic Co-operation and Development, calling for a new multi-lateral treaty between governments on the safe development of superintelligence and the establishment of an interparliamentary union on AI.
Meanwhile, government bodies continue evaluating frontier models. A Cabinet Office spokesperson noted that the UK is a world leader in AI security and that the country’s security institute collaborates closely with industry partners, pointing to recent testing of powerful models prior to public release.
Financial Stakes and Amazon’s Multi-Billion-Dollar Infrastructure Bet
These existential safety debates directly intersect with massive commercial commitments. Amazon CEO Andy Jassy detailed extensive corporate alignments with leading developers during an earnings call, noting that Anthropic and OpenAI have made multi-year, multi-gigawatt commitments to Amazon’s custom Trainium chips.
Anthropic has anchored Project Rainier by committing to secure up to 5 gigawatts of compute capacity utilizing over 500,000 Trainium2 chips. The financial footprint of this partnership is immense; Amazon’s stake in Anthropic produced a $53.4 billion one-time non-operating pre-tax gain in the second quarter of fiscal 2026, propelling GAAP earnings per share to $5.75 against consensus expectations.
The partnership fuels substantial growth within Amazon Web Services, which posted year-over-year revenue growth of 36.7 percent—its fastest pace in 18 quarters—while overall company backlog reached $496 billion. At the same time, Jassy confirmed that we are pursuing our own frontier model to ensure competitive positioning as multiple equivalent models enter bedrock infrastructure. With cash capital expenditures hitting $53.1 billion in the second quarter alone, the philosophical debate over machine safety has transformed into a measurable financial factor for institutional investors.
