Astrophysicist Paul Sutter experienced a severe presentation mishap in February when a collaborator pointed out fundamental errors in an algorithm he updated using generative AI.
Generative artificial intelligence has firmly entrenched itself across academic and technical disciplines.
The Presentation Mishap That Exposed AI Coding Flaws
The hazard hit home for astrophysicist and science communicator Paul Sutter during a February presentation in front of a room full of collaborators describing his updated void-spotting algorithm. Sutter had relied heavily on AI coding tools to rewrite the program, a practice colloquially known as vibe coding. He previously found the iterative chat sessions indispensable for generating code that was ten times faster than the old one
while handling surveys a hundred times larger
through a more sophisticated way of dealing with the ugly realities of an actual data set
.
Just ten minutes into the demonstration, a collaborator interrupted the speaker.
“something seemed off to them.”
Paul Sutter, via Futurism
The edge cases of the surveys had been handled incorrectly by the AI-generated code. Sutter later noted that the error wasn’t a typo, and it wasn’t a missing citation or a factor of two
. Instead, the logic was subtly and thoroughly flawed, compromising every calculation downstream from the mistake.
Moving Beyond Classical Vibe Coding Into Quantum Computing
While Sutter’s experience serves as a warning in classical astrophysics, the practice of prompting AI to write software is expanding into even more complex domains. Researchers at Pasqal, a quantum-computing startup based in Paris, developed an artificial-intelligence tool designed to let users translate English-language prompts directly into quantum-computing code and autonomously execute it.
Quantum programming traditionally demands sophisticated skills involving both quantum physics and advanced algorithm design. The Pasqal agent attempts to lower this barrier by converting complex physical simulations into machine-executable instructions. Co-founder Christophe Jurczak reported that the agent allowed him to run experiments from his couch in Dallas, Texas, that would normally require a team of specialized physicists.
How Automated Quantum Simulation Agents Operate
To test the limits of frontier large language models in quantum workflows, the Pasqal team focused on quantum simulations. These calculations tune a quantum computer to mimic physical systems, such as a catalyst or materials exhibiting unusual magnetic properties. Such tasks would overwhelm classical computing architectures.
The research team subjected their AI agent to three distinct evaluations based on published physics papers. Two of those tests required modeling materials where atoms flip up or down like tiny bar magnets depending on their neighbors’ orientation. Chief technology officer Loïc Henriet noted that this translation step typically demands expertise combining material physics and quantum programming. Before executing code on physical quantum hardware located in Sherbrooke, Canada, or Dhahran, Saudi Arabia, the AI agent first validates the instructions using virtual simulation environments running on classical computers.
Understanding the Limitations of Next-Word Predictors
Sutter described modern large language models as little more than sophisticated next-word predictors that generate fluent text because human training intentionally selected for conversational charm.
“It’s a trait we bred, the way we bred wolves into dogs that watch our faces when we open the treat bag.”
Paul Sutter, via Futurism
When tested with automated evaluation utilities such as the Pangram AI detecting tool—which noted that 56 percent of Sutter’s latest piece for Nautilus appeared to be written by an AI—analysts note that distinguishing human expertise from algorithmic synthesis remains challenging across academic publishing.
Auditing the Chain of Reasoning in Scientific Work
The broader scientific community faces a reckoning over accountability as poorly researched or unedited AI slop infiltrate research papers. Researchers are ultimately responsible for everything they publish, meaning subtle hallucinations can ruin months of collaborative effort before anyone detects the underlying fault.
Sutter likened our current reliance on generative models to alchemy, warning that researchers must meticulously audit every output rather than trusting conversational fluency. Following his February presentation error, the astrophysicist altered his workflow to immediately distrust unverified machine outputs.
