AI Chatbots Rarely Withheld Needed Care in a Choosing Wisely Test. They Over-Ordered EKGs and Thyroid Ultrasounds.
Six AI models recommended low-value care in 5.4 percent of responses and withheld indicated care in 0.6 percent, an Annals study found.
Six commercial large language models recommended unnecessary tests or treatments in 5.4 percent of responses to Choosing Wisely scenarios and withheld indicated care in 0.6 percent, according to a study published Oct. 20 in Annals of Internal Medicine.
The study, "Overuse and False Restraint in Large Language Model Recommendations for Choosing Wisely Scenarios", was led by Dean Ariel of Clalit Health Services and Guy Laban of Ben-Gurion University of the Negev, with Stav Endelman of Rabin Medical Center, according to the university's announcement. Choosing Wisely is the campaign in which specialty societies list tests and treatments that are often unnecessary.
How was the test built?
The researchers wrote 225 clinical scenarios based on 15 Choosing Wisely recommendations and put them to six models from OpenAI, Anthropic and Google, according to the study abstract. Each recommendation was paired with cases where the intervention was redundant and closely matched control cases where it was needed. That design measures two errors: overuse, recommending low-value care, and false restraint, withholding care that is indicated. The announcement does not name the specific model versions.
What did the models get wrong?
Overuse was the more common error, and it was concentrated. Eight of the 15 recommendations drew no unnecessary recommendations at all. Two accounted for most of the overuse: thyroid ultrasound for a subcentimeter incidental nodule, recommended in 42.8 percent of responses on that recommendation, and routine electrocardiography in low-risk adults, in 28.3 percent, according to the announcement. False restraint was rare and highest for prostate-specific antigen screening, the abstract says.
Models also disagreed with each other in gray areas. On identical cases, one model recommended an unnecessary test under 10 percent of the time and another about 80 percent of the time, the announcement says.
The models' errors leaned toward doing more. That is the direction that adds incidentalomas, follow-up visits and patient costs, and it is the direction a worried patient is least likely to question.
How large is the imbalance?
By The Script Pad's arithmetic from the reported rates, overuse was nine times as common as false restraint (5.4 versus 0.6 percent), and the spread between the most and least test-prone models on the same gray-area cases was at least eightfold. The authors' explanation is about training data. "Language models learned medicine from human clinical literature—including our less disciplined habits," Ariel said in the announcement. Laban added that "language models tend to go along with the person in front of them."
What are the limits?
This was a benchmark with single prompts and no follow-up questions, not observation of real patients or clinicians, and the authors note the results may not carry over to consumer use. Model versions change quickly, so the rates describe the systems tested, not today's products. The full paper is behind a subscription; per-model results were not in the abstract or announcement.
What it means in clinic
Patients increasingly arrive having asked a chatbot what they need. This study suggests the likeliest error is a suggestion to add a test, especially an EKG or a thyroid ultrasound, rather than to skip one. When a patient asks for a test a chatbot recommended, ask what they were told and walk through the indication, as with any other outside advice. Practices using AI tools for drafting orders or patient portal replies should keep a clinician review step, and those still choosing tools can test them on their own low-value care scenarios first, as described in our guide to choosing AI tools.
Reported, fact-checked and edited by The Script Pad's editorial staff. See How we use AI.
This article is for professional education and does not replace clinical judgment. Treatment decisions should be based on the individual patient and current guidelines.
