ChatGPT Capable of Clinical Reasoning -- Maybe Better Than Clinicians
By Lori Solomon HealthDay Reporter
THURSDAY, April 4, 2024 -- A chatbot outperforms physicians in clinical reasoning, according to a research letter published online April 1 in JAMA Internal Medicine.
Stephanie Cabral, M.D., from the Beth Israel Deaconess Medical Center in Boston, and colleagues compared a large language model’s reasoning abilities against human performance using standards developed for physicians. Responses were compared for selected cases queried in GPT-4 (OpenAI) in August 2023 and from 21 internal medicine attending physicians and 18 residents.
The researchers found that median Revised-IDEA (R-IDEA) scores were 10 (range, 9 to 10) for chatbot, 9 (6 to 10) for attendings, and 8 (4 to 9) for residents. The chatbot had a significantly higher estimated probability of achieving high R-IDEA scores than attendings and residents and had significantly higher R-IDEA scores than attendings and residents. There were no significant differences in attendings’ and residents’ scores. For diagnostic accuracy, the chatbot performed similarly to attendings and residents. Scores were also similar for correct clinical reasoning and cannot-miss diagnosis inclusion. However, the chatbot had more frequent instances of incorrect clinical reasoning (13.8 percent) than residents (2.8 percent) but not attendings (12.5 percent).
"There are multiple steps behind a diagnosis, so we wanted to evaluate whether large language models are as good as physicians at doing that kind of clinical reasoning," coauthor Adam Rodman, M.D., also from Beth Israel, said in a statement. "It's a surprising finding that these things are capable of showing the equivalent or better reasoning than people throughout the evolution of clinical case."
Two authors disclosed ties to industry.
Abstract/Full Text (subscription or payment may be required)
Disclaimer: Statistical data in medical articles provide general trends and do not pertain to individuals. Individual factors can vary greatly. Always seek personalized medical advice for individual healthcare decisions.
![](/img/logo/vendor/healthday-logo.png)
© 2024 HealthDay. All rights reserved.
Posted April 2024
Read this next
AI-Assisted Contours Superior to Cognitively Defined Prostate Cancer Contours
WEDNESDAY, July 3, 2024 -- Artificial intelligence (AI)-assisted definition of prostate cancer contours reduces underestimation of the extent of prostate cancer, according to a...
Mean Cost of Bringing New Drug to U.S. Market Is $879.3 Million
TUESDAY, July 2, 2024 -- The mean cost of developing a new drug for the U.S. market is estimated to be $879.3 million when both drug development failure and capital costs are...
Patient–Primary Care Provider Language Concordance Tied to Better Outcomes
TUESDAY, July 2, 2024 -- Patient-family physician language concordance is associated with a lower risk for adverse outcomes, according to a study published online June 3...
More news resources
- FDA Medwatch Drug Alerts
- Daily MedNews
- News for Health Professionals
- New Drug Approvals
- New Drug Applications
- Drug Shortages
- Clinical Trial Results
- Generic Drug Approvals
Subscribe to our newsletter
Whatever your topic of interest, subscribe to our newsletters to get the best of Drugs.com in your inbox.