ChatGPT Only Gets Diagnoses Correct Half of the Time
By Lori Solomon HealthDay Reporter
THURSDAY, Aug. 8, 2024 -- ChatGPT is not accurate as a diagnostic tool, but does offer some medical educational benefits, according to a study published online July 31 in PLOS ONE.
Ali Hadi, from the Schulich School of Medicine and Dentistry at Western University in London, Ontario, Canada, and colleagues investigated ChatGPT’s diagnostic accuracy and utility in medical education. The analysis included 150 Medscape case challenges (September 2021 to January 2023) that were inputted into ChatGPT.
The researchers found that ChatGPT answered 49 percent of cases correctly, with an overall accuracy of 74 percent, a precision of 48.67 percent, sensitivity of 48.67 percent, specificity of 82.89 percent, and an area under the curve of 0.66. ChatGPT struggled with the interpretation of laboratory values and imaging results, but was generally correct in ruling out a specific differential diagnosis and providing reasonable next diagnostic steps. Just over half of answers were complete and relevant (52 percent) and a similar percentage of answers were characterized as low cognitive load (51 percent).
"Additional research should focus on enhancing the accuracy and dependability of ChatGPT as a diagnostic instrument," the authors write. "Integrating ChatGPT into medical education and clinical practice necessitates a thorough examination of its educational and clinical limitations. Transparent guidelines should be established for ChatGPT’s clinical usage, and medical students and clinicians should be trained on how to effectively and responsibly employ the tool."
Disclaimer: Statistical data in medical articles provide general trends and do not pertain to individuals. Individual factors can vary greatly. Always seek personalized medical advice for individual healthcare decisions.
© 2024 HealthDay. All rights reserved.
Posted August 2024
Read this next
Casual Blood Glucose Testing Misses ~70 Percent of Gestational Diabetes Cases
FRIDAY, Sept. 27, 2024 -- Casual blood glucose (CBG) testing at 24 to 28 gestational weeks misses about 70 percent of women with gestational diabetes mellitus (GDM), according to...
Large Difference in Survival Seen for Preemies Born at 25 Versus 22 Weeks
FRIDAY, Sept. 27, 2024 -- For infants born prematurely, survival increases considerably for those born at 25 versus 22 gestational weeks, according to a study published online...
Adverse Brain Health Outcomes Increased for Sexual, Gender Minority Groups
FRIDAY, Sept. 27, 2024 -- Sexual and gender-minority (SGM) persons have increased odds of adverse brain health outcomes, according to a study published online Sept. 25 in...
More news resources
- FDA Medwatch Drug Alerts
- Daily MedNews
- News for Health Professionals
- New Drug Approvals
- New Drug Applications
- Drug Shortages
- Clinical Trial Results
- Generic Drug Approvals
Subscribe to our newsletter
Whatever your topic of interest, subscribe to our newsletters to get the best of Drugs.com in your inbox.