Doctors and Nurses Outperform AI in Emergency Room Triage, study finds
A new study reveals that while artificial intelligence shows promise, human clinicians – emergency medicine doctors and nurses – remain more accurate in triaging patients in the emergency department.The research, presented in September at the European Emergency Medicine Congress in Vienna, underscores the critical role of experienced medical judgment even as AI tools rapidly evolve.
the increasing strain on emergency departments, coupled with a growing workload for nurses, prompted researchers to investigate whether AI could alleviate some of the pressure. The study,conducted at Vilnius University Hospital Santaros Klinikos in Lithuania,aimed to determine if AI could support triage decision-making,improve efficiency,and reduce staff burden.
researchers distributed a paper and digital questionnaire to six emergency medicine doctors and 51 nurses. Participants were tasked with classifying the urgency of clinical cases randomly selected from 110 reports in the PubMed database, utilizing the Manchester Triage System – a standardized method for categorizing patients based on the severity of their condition.The same cases were then analyzed by ChatGPT version 3.5. A high participation rate was observed, with 100% of doctors (6) and 86.3% of nurses (44) completing the questionnaire.
“We conducted this study to address the growing issue of overcrowding in the emergency department and the escalating workload of nurses,” a senior researcher explained. “Given the rapid development of AI tools like ChatGPT, we aimed to explore whether AI could support triage decision-making, improve efficiency and reduce the burden on staff in emergency settings.”
The findings demonstrated that, AI underperformed compared to both nurses and doctors. AI achieved an overall accuracy of 50.4%, while nurses scored 65.5% and doctors reached 70.6%.Sensitivity – the ability to correctly identify truly urgent cases – was also lower for AI at 58.3%, compared to 73.8% for nurses and 83.0% for doctors. Across all urgency categories analyzed, doctors consistently achieved the highest scores.
However, the study revealed a surprising nuance: AI outperformed nurses specifically in identifying the most urgent, life-threatening cases. AI’s accuracy in this highest triage category was 27.3% compared to 9.3% for nurses, and its specificity – the ability to correctly identify cases not requiring immediate intervention – was 27.8% versus 8.3% for nurses.
“These results suggest that while AI generally tends to over-triage, it may be somewhat more cautious in flagging critical cases, which can be both a strength and a drawback,” the researcher noted. “While we anticipated that AI might not outperform experienced clinicians and nurses, we were surprised that in some areas AI performed quite well. Actually, in the most urgent triage category, it demonstrated higher accuracy than nurses.”
The researchers emphasize that AI shoudl not be used as a standalone triage tool, but rather as a decision-support tool to augment, not replace, clinical judgment. AI could potentially assist in consistently prioritizing the most urgent cases and supporting less experienced staff, but careful integration and human oversight are crucial to avoid inefficiencies caused by excessive triaging.
“Hospitals should approach AI implementation with caution and focus on training staff to critically interpret AI suggestions,” the researcher concluded. Future studies are planned to evaluate newer AI versions and models specifically fine-tuned for medical applications, with larger participant groups, incorporating ECG interpretation, and exploring AI’s role in managing mass casualty incidents and nurse training.
