AI Screens Patients for Clinical Trials With Greater Accuracy Than Staff
By Lori Solomon HealthDay Reporter
THURSDAY, June 20, 2024 -- Large language model-based solutions can enhance clinical trial screening performance and reduce costs by automating the screening process, according to a study published online June 17 in NEJM AI.
Ozan Unlu, M.D., from Brigham and Women's Hospital in Boston, and colleagues evaluated the utility of a Retrieval-Augmented Generation (RAG)-enabled GPT-4 system to improve the accuracy, efficiency, and reliability of screening for a trial involving patients with symptomatic heart failure. Findings were based on clinical notes from 100, 282, and 1,894 patients for development, validation, and test datasets, respectively.
The researchers reported that answers from the RAG-Enabled Clinical Trial Infrastructure for Inclusion Exclusion Review (RECTIFIER), a clinical note-based, question-answering system powered by RAG and GPT-4, closely aligned with the expert clinicians' answers across the target criteria, with accuracy ranging from 97.9 to 100 percent (Matthews correlation coefficient [MCC], 0.837 and 1) for RECTIFIER versus 91.7 and 100 percent (MCC, 0.644 and 1) for the study staff. Compared with study staff, RECTIFIER performed better at determining symptomatic heart failure (accuracy 91.7 and 97.9 percent, respectively; MCC, 0.721 and 0.924). With RECTIFIER, the sensitivity and specificity for determining patient eligibility were 92.3 and 93.9 percent, respectively, versus 90.1 and 83.6 percent, respectively, for the study staff. The single-question approach to determining eligibility with RECTIFIER resulted in an average cost of 11 cents per patient, and the combined-question approach resulted in an average cost of 2 cents per patient.
"Integrating such technologies requires careful consideration of potential hazards and should include safeguards such as final clinician review," the authors write.
Several authors disclosed ties to the pharmaceutical industry.
Disclaimer: Statistical data in medical articles provide general trends and do not pertain to individuals. Individual factors can vary greatly. Always seek personalized medical advice for individual healthcare decisions.
© 2024 HealthDay. All rights reserved.
Posted June 2024
Read this next
Survival Improves With Open Hysterectomy for Cervical Cancer
FRIDAY, June 28, 2024 -- For patients with early-stage cervical cancer, disease-free and overall survival are lower for patients undergoing minimally invasive versus open radical...
Precision-Guided Treatment Improves Outcomes for High-Risk Pediatric Cancer
FRIDAY, June 28, 2024 -- Precision-guided treatment improves outcomes for children with high-risk cancer, according to a study published online June 6 in Nature Medicine. Loretta...
Recommendations Issued for Advanced Imaging for Pediatric Patients in ED
THURSDAY, June 27, 2024 -- In a policy statement issued by the American Academy of Pediatrics and published online June 27 in Pediatrics, recommendations are presented for...
More news resources
- FDA Medwatch Drug Alerts
- Daily MedNews
- News for Health Professionals
- New Drug Approvals
- New Drug Applications
- Drug Shortages
- Clinical Trial Results
- Generic Drug Approvals
Subscribe to our newsletter
Whatever your topic of interest, subscribe to our newsletters to get the best of Drugs.com in your inbox.