A former Anthropic researcher warned that AI development could lead to human extinction, sparking debate among industry leaders and political figures about the technology’s risks and future.
Jacob Coxon, a 27-year-old AI researcher who previously worked at Anthropic and OpenAI, told the BBC that people in the field are genuinely frightened
about the pace of AI advancements. His resignation post, which went viral on X, raised concerns about AI’s potential to autonomously produce deadly viruses
or hack into critical infrastructure that the world runs on.
Coxon’s warnings align with Dario Amodei, who in an essay on Saturday argued for development to slow down.
The Resignation and the Warnings
Coxon’s departure from Anthropic was driven by his belief that companies are racing toward self-improving superintelligence
without adequate safeguards. Neither company is acting responsibly,
he wrote in a viral X thread. They are racing to self-improving superintelligence and gambling with our lives.
His post, which amassed over 156 million views, described scenarios where AI systems could autonomously conduct cyberattacks or weaponize biological threats. Any kind of concrete scenario you can lay out ends up sounding like science fiction,
Coxon admitted, but he argued that past AI breakthroughs—such as the recent OpenAI incident where systems hacked Hugging Face—demonstrate the plausibility of such risks.
Industry Reactions and Controversies
Hugging Face CEO Clement Delangue dismissed the concerns as like asking your AC guy about climate change,
arguing that the focus on existential risks distracts from more immediate challenges. The world is in peril,
wrote former Anthropic safety lead Mrinank Sharma in a February resignation post, echoing Coxon’s sentiments.
Meanwhile, Nvidia boss Jensen Huang called Coxon’s claims complete nonsense,
stating that AI is going to be the end of humanity
is complete nonsense.
Huang’s remarks reflect a broader pushback in Silicon Valley against what some view as exaggerated fears. However, Coxon and his allies argue that the stakes are too high to ignore. Every one of these companies believes slowing down hands the lead to someone less careful,
said Shama Hyder, an AI professor at Link School of Business. Even the people who understand the risk have a reason to accelerate.
Political and Public Response
The debate has spilled into politics, with Illinois Gov. JB Pritzker responding to Coxon’s post on X. It’s becoming more clear the threat AI poses to humanity, so I’m calling for immediate action from the industry and Washington,
Pritzker wrote. His statement underscores growing pressure on regulators to address AI risks as companies like Anthropic and OpenAI prepare for potential stock market debuts.
Coxon’s resignation has also reignited discussions about the ethical responsibilities of AI developers. She later posted a poem titled How We Programmed the Apocalypse,
reflecting the unease within the field. The dangers of contributing to the technology’s accelerating pace outweighed any financial gains,
Coxon told Axios, noting he left before his equity vested.
The coming months will test whether industry leaders can balance innovation with safety. Anthropic and OpenAI are expected to launch record-setting IPOs, but critics argue that the rush to market may prioritize speed over safeguards. Meanwhile, the debate over AI’s risks will likely intensify as researchers, policymakers, and the public grapple with the technology’s implications. If we don’t slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future,
Coxon warned—a statement that continues to divide the AI community.
