Google’s artificial intelligence-powered search summaries are increasingly relied upon for quick answers, but a growing chorus of experts is raising concerns that the technology is putting users at risk by downplaying crucial health disclaimers. The issue centers on Google’s AI Overviews, which appear at the top of search results, and the way—or rather, the lack of a way—the company alerts users to the potential for inaccurate medical information. This has led to calls for more prominent warnings and a reevaluation of how Google prioritizes speed over accuracy in sensitive areas like healthcare.
The core of the problem, as detailed in recent investigations, is that Google doesn’t consistently display disclaimers when initially presenting medical advice through AI Overviews. Whereas the company states that its AI will “inform people when it’s vital to seek out expert advice or to verify the information presented,” these warnings are often hidden behind an additional click—the “Show more” button—and appear in a smaller font size at the bottom of the expanded summary. This delayed and subtle presentation of risk information is prompting fears that users may mistakenly believe the AI-generated advice is definitive and reliable.
The concerns aren’t theoretical. Earlier this year, a Guardian investigation revealed instances of inaccurate health information being presented in AI Overviews, including advice to pancreatic cancer patients to avoid high-fat foods—a recommendation directly contrary to established medical guidance. The investigation also highlighted misleading information regarding liver function tests and women’s cancer screenings. Following that reporting, Google removed AI Overviews for some, but not all, medical searches.
The Design Problem: Speed Over Safety?
Experts argue that the design of AI Overviews inherently prioritizes speed and convenience over accuracy, creating a dangerous dynamic when dealing with health information. Gina Neff, a professor of responsible AI at Queen Mary University of London, described the issue as “by design,” stating that “AI Overviews are designed for speed, not accuracy, and that leads to mistakes in health information, which can be dangerous.” The immediate presentation of a concise summary, while appealing to users seeking quick answers, can create a false sense of reassurance and discourage further investigation.
Sonali Sharma, a researcher at Stanford University’s centre for AI in medicine and imaging (AIMI), echoed this sentiment, explaining that the placement of AI Overviews at the top of search results often leads users to accept the information as complete. “For many people, due to the fact that that single summary is there immediately, it basically creates a sense of reassurance that discourages further searching, or scrolling through the full summary and clicking ‘Show more’ where a disclaimer might appear,” Sharma said. The risk is compounded by the potential for AI to generate “partially correct and partially incorrect information,” making it difficult for even informed individuals to discern accuracy.
Why Prominent Disclaimers Matter
The absence of upfront disclaimers isn’t simply a matter of transparency; it fundamentally alters the way users interact with health information, according to Pat Pataranutaporn, an assistant professor and researcher at MIT specializing in AI and human-computer interaction. “The absence of disclaimers when users are initially served medical information creates several critical dangers,” Pataranutaporn explained. “First, even the most advanced AI models today still hallucinate misinformation or exhibit sycophantic behaviour, prioritising user satisfaction over accuracy. In healthcare contexts, this can be genuinely dangerous.”
Pataranutaporn further emphasized the human element, noting that users may not always articulate their symptoms accurately or provide sufficient context for the AI to generate reliable advice. Disclaimers, she argues, serve as a crucial “intervention point,” disrupting the automatic trust users might place in the AI and prompting them to critically evaluate the information received. Tom Bishop, head of patient information at Anthony Nolan, a blood cancer charity, agreed, stating that a disclaimer “needs to be much more prominent…right at the top” and “the same size font as everything else.”
Google’s Response and Ongoing Concerns
A Google spokesperson defended the current approach, stating that it’s “inaccurate to suggest that AI Overviews don’t encourage people to seek professional medical advice.” The spokesperson added that AI Overviews “frequently mention seeking medical attention directly within the overview itself, when appropriate.” Yet, critics argue that this is insufficient, as the primary issue is the lack of a clear and immediate warning before users are exposed to potentially inaccurate information.
The debate over AI-generated health information extends beyond Google. As AI becomes increasingly integrated into healthcare, ensuring the accuracy and reliability of these tools is paramount. The current situation highlights the need for careful consideration of how AI is designed and deployed in sensitive areas, and the importance of prioritizing user safety over speed and convenience.
What’s Next for AI and Healthcare?
The pressure on Google to address these concerns is likely to intensify. The company is currently evaluating feedback and iterating on its AI Overviews, but a more substantial overhaul of the disclaimer system may be necessary to regain public trust. The broader implications of this situation extend to the entire AI industry, raising questions about the ethical responsibilities of developers and the need for greater regulatory oversight. Google has not announced a specific timeline for further changes to AI Overviews, but the company has indicated it will continue to monitor the technology’s performance and address any identified issues.
This is a developing story, and we will continue to provide updates as they become available. Share your thoughts and experiences with AI-powered search in the comments below.
