Artificial intelligence is showing promise in helping doctors decide how to treat liver cancer, but its advice isn’t always reliable-especially as the disease becomes more advanced.That’s the key takeaway from a new study examining the performance of several large language models.
AI shows Potential, But Isn’t Ready to Replace Doctors in Liver Cancer Treatment
A new retrospective study reveals AI can aid in early-stage liver cancer decisions, but struggles with complex, late-stage cases.
- Large language models (LLMs) like Gemini 2.0, ChatGPT 4o, and Claude 3.5 can offer treatment suggestions for early-stage liver cancer that align with medical guidelines.
- The accuracy of these AI recommendations decreases as the cancer progresses to later stages.
- Physicians prioritize a patient’s overall liver function, while LLMs tend to focus more on the characteristics of the tumor itself.
- Researchers emphasize that AI should be used as a tool to *support* clinical expertise, not replace it.
For patients with straightforward cases of early-stage hepatocellular carcinoma, commonly used large language models (LLMs) were able to provide treatment recommendations consistent with established guidelines, according to findings from a retrospective registry study conducted in South Korea and published in PLOS Medicine.
“Our study shows that [LLMs] can help support treatment decisions for early-stage liver cancer, but their performance is more limited in advanced disease,” said study author Dr. Jihoon Kim, of the department of Liver and Biliary-Pancreatic Surgery at Samsung Medical Center in Seoul, South Korea.
A disease (early stage) experienced better survival rates when the LLM recommendations aligned with the physicians’ treatment plans (ChatGPT 4o hazard ratio [HR] = 0.743; 95% confidence interval [CI] = 0.665-0.831; P < .001). Though, for patients with BCLC-C disease (advanced stage), agreement between AI recommendations and physician treatments was linked to *worse* survival outcomes (chatgpt 4o HR = 1.650; 95% CI = 1.523-1.787; P < .001; Gemini 2.0 HR = 1,586; 95% CI = 1,470-1,711; P < .001; Claude 3.5 HR = 1.483; 95% CI = 1.366-1.610; P < .001). Patients with BCLC-B disease showed only modest or insignificant associations between AI alignment and survival.
Can AI predict liver cancer treatment outcomes? The study suggests AI can be helpful for early-stage cases,but its predictions become less reliable as the disease progresses,and may even indicate poorer outcomes when followed in advanced cases.
Researchers found that physicians placed greater emphasis on liver function parameters, while LLMs focused more on tumor characteristics. Physicians frequently enough avoided curative treatments if a patient’s liver function was compromised in early-stage cases,but opted for more localized therapies for advanced-stage disease even when liver function was good-choices that sometimes deviated from guideline recommendations for systemic therapy.
“While llms may serve as adjunctive tools for guideline-concordant decisions in straightforward scenarios,their recommendations may reflect limited contextual awareness in complex clinical situations requiring individualized care,” the study authors wrote.”LLM recommendations should be interpreted cautiously alongside clinical judgment.”
The study’s retrospective nature, lack of imaging data, and focus on treatments aligned with older guidelines mean the findings should be validated in future prospective studies, according to the researchers.
This work was supported by the National Research Foundation of Korea grant funded by the Korea government funded by the Ministry of Health & welfare,Republic of Korea.
Worth a look
