ChatGPT Shows Poor Performance in Answering Drug-Related Questions
By Lori Solomon HealthDay Reporter
TUESDAY, Dec. 12, 2023 -- ChatGPT provided no response or incomplete or wrong answers to nearly three-quarters of drug-related questions reviewed by pharmacists, according to a study presented at the American Society of Health-System Pharmacists Midyear Clinical Meeting, held from Dec. 3 to 7 in Anaheim, California.
Sara Grossman, Pharm.D., from Long Island University in New York, and colleagues asked the publicly available, free version of ChatGPT 39 questions that had been posed to a drug information service. Two investigators compared ChatGPT responses to benchmark responses based on a literature search.
Of the 39 questions, the researchers reported that ChatGPT provided responses for 72 percent. Just over one-third of ChatGPT responses (36 percent) were accurate and complete and had no irrelevant information, whereas 64 percent had inaccurate or incomplete responses. One-third of the complex questions and 23 percent of the noncomplex questions were answered in a satisfactory manner.
"Health care professionals and patients should be cautious about using ChatGPT as an authoritative source for medication-related information," Grossman said in a statement. "Anyone who uses ChatGPT for medication-related information should verify the information using trusted sources."
Disclaimer: Statistical data in medical articles provide general trends and do not pertain to individuals. Individual factors can vary greatly. Always seek personalized medical advice for individual healthcare decisions.
© 2024 HealthDay. All rights reserved.
Posted December 2023
Read this next
AI-Assisted Contours Superior to Cognitively Defined Prostate Cancer Contours
WEDNESDAY, July 3, 2024 -- Artificial intelligence (AI)-assisted definition of prostate cancer contours reduces underestimation of the extent of prostate cancer, according to a...
Mean Cost of Bringing New Drug to U.S. Market Is $879.3 Million
TUESDAY, July 2, 2024 -- The mean cost of developing a new drug for the U.S. market is estimated to be $879.3 million when both drug development failure and capital costs are...
Patient–Primary Care Provider Language Concordance Tied to Better Outcomes
TUESDAY, July 2, 2024 -- Patient-family physician language concordance is associated with a lower risk for adverse outcomes, according to a study published online June 3...
More news resources
- FDA Medwatch Drug Alerts
- Daily MedNews
- News for Health Professionals
- New Drug Approvals
- New Drug Applications
- Drug Shortages
- Clinical Trial Results
- Generic Drug Approvals
Subscribe to our newsletter
Whatever your topic of interest, subscribe to our newsletters to get the best of Drugs.com in your inbox.