The rise of artificial intelligence in healthcare is presenting both opportunities and risks, as more individuals turn to AI-powered chatbots for medical information. Companies like OpenAI and Anthropic are at the forefront of this trend, offering tools that analyze personal health data – from wearable devices to electronic health records – to provide insights and support. However, experts caution that these technologies are not substitutes for professional medical advice and raise significant privacy concerns.
In January, OpenAI launched ChatGPT Health, a service capable of analyzing users’ health documentation, wellness app data, and measurements from wearable devices. Anthropic quickly followed suit with similar functionality through its Claude chatbot. Both companies emphasize that these large language models (LLMs) are not intended to replace medical care or provide diagnoses, but rather to assist in interpreting complex medical information, preparing for medical appointments, and identifying important health trends from available datasets. This growing trend of using AI for health questions is prompting a wider conversation about the role of technology in personal healthcare.
One key advantage of these fresh chatbots is their ability to consider a user’s medical history – including medications, age, and previous medical records – when formulating responses. Robert Wachter, a specialist in medical technology at the University of California, San Francisco (UCSF), notes that this personalization results in more tailored answers than a simple internet search. “The alternative is often nothing, or the patient relying on their own instincts,” Wachter said, advocating for responsible use and providing as much detail as possible when interacting with these systems.
However, the potential for data privacy breaches is a serious concern. In the United States, the Health Insurance Portability and Accountability Act (HIPAA) strictly regulates how doctors, hospitals, and insurers handle patient data. However, as noted by Anthropic in a recent blog post, these regulations do not currently apply to the companies developing the chatbots themselves. “When someone uploads their medical history into a large language model, it’s very different than handing it to a new doctor,” warned Lloyd Minor, dean of the Stanford University School of Medicine.
OpenAI and Anthropic state that health data is stored separately and is not used to train their models, and users can revoke their consent at any time. Despite these assurances, the lack of direct HIPAA oversight raises questions about data security and potential misuse.
Independent testing of these technologies is still in its early stages, and initial results are mixed. A recent study by the University of Oxford, involving 1300 participants, found that those who used chatbots to gather health information did not make better decisions than those who relied on internet searches or their own judgment. The study, as reported by TechCrunch, revealed that although chatbots correctly identified diseases in 95% of detailed case studies, their performance significantly declined when interacting with real users.
Adam Mahdi, who led the Oxford study, attributed this discrepancy to users often failing to provide sufficient information. This led to chatbots offering a mix of correct and incorrect answers, which users struggled to differentiate. The study highlights the importance of comprehensive data input for accurate AI-driven health insights.
Wachter believes a crucial area for chatbot development is the ability to ask targeted follow-up questions to elicit essential details from users, mirroring the approach of a physician during a consultation. He advises individuals using chatbots for health inquiries to utilize multiple systems concurrently, as a form of “second opinion.” He strongly emphasizes that immediate medical attention is necessary for urgent symptoms such as shortness of breath, chest pain, or severe headache.
Expanding Capabilities: Claude for Healthcare and Beyond
Anthropic is expanding its offerings with Claude for Healthcare, a set of tools designed for healthcare providers, payers, and health tech companies. This includes “connectors” providing access to platforms and databases like the Centers for Medicare and Medicaid Services (CMS) Coverage Database, the International Classification of Diseases, 10th Revision (ICD-10), the National Provider Identifier Standard, and PubMed. These tools aim to streamline administrative tasks, such as prior authorization reviews, which often consume significant clinician time.
According to Anthropic, Claude for Healthcare could expedite prior authorization review, a process where doctors must submit additional information to insurance providers to determine coverage for medications or treatments. Mike Krieger, Anthropic’s Chief Product Officer, noted that clinicians often spend more time on paperwork than direct patient care, making this an ideal area for automation.
Performance Improvements in Medical and Life Sciences
Recent advancements in Anthropic’s models, particularly Claude Opus 4.5, demonstrate significant improvements in performance on medical and scientific tasks. Evaluations show improvements in accuracy against internal benchmarks for key life sciences tasks, including scientific figure interpretation, computational biology, and protein understanding. Claude 4.5 models, evaluated with extended thinking (64k tokens) and native tool use, also showed promising results in spatial biology analysis using LatchBio’s SpatialBench.
Key Performance Metrics
- SpatialBench: Spatial biology analysis by LatchBio
- MedCalc: Medical calculation accuracy (with Python code execution)
- MedAgentBench: Medical agent task completion (Stanford)
These advancements suggest that AI is becoming increasingly capable of assisting in complex medical and scientific research, but the need for careful validation and responsible implementation remains paramount.
As AI continues to integrate into healthcare, ongoing research and careful consideration of ethical and privacy implications will be crucial. The next step in the evolution of these technologies will likely involve further refinement of their ability to gather comprehensive patient information and provide accurate, reliable insights, while adhering to stringent data protection standards.
What are your thoughts on the use of AI in healthcare? Share your comments below and join the conversation.
