1. A cross-over study compared clinicians’ electroencephalogram (EEG) interpretation accuracy with artificial intelligence (AI) assistance against the same participants without AI assistance.
2. The performance of all clinicians was significantly higher with AI assistance as compared to without (71% vs 47%).
Evidence Rating Level: 2 (Good)
Study Rundown: Seizures are serious medical events that increase the risk of permanent disability or death. However, clinical interpretations of seizures with electroencephalography (EEG) are hampered by clinician availability and subjectivity. Barnett and colleagues developed a novel deep-learning algorithm called ProtoPMed-EEG trained on data from 2711 hospitalized patients. The AI model underwent a two-stage, multiuser study with a cohort of clinical practitioners without expertise in machine learning. The clinicians were separated into two groups randomly, with each group given ProtoPMed-EEG at different stages, two weeks apart. Their diagnostic accuracy with and without AI assistance was compared using 100 EEG samples. The study found that mean user diagnostic accuracy was higher with AI assistance for every clinician as compared to without. The mean inter-rater reliability similarly improved. Additionally, most users believed their diagnostic ability improved after completing the stage with AI. Overall, this study demonstrated AI’s ability to assist clinicians in making superior diagnoses and may play future roles in diagnostic assistance and clinical education.
Click here to read the study in NEJM AI
Relevant Reading: Automated Interpretation of Clinical Electroencephalograms Using Artificial Intelligence
In-Depth [randomized controlled trial]: Barnett and colleagues developed ProtoPMed-EEG using 50,697 EEG samples collected from 2711 ICU patients between July 2006 and March 2020. The training samples were grouped into one of the following categories by 124 EEG raters: seizure, lateralized periodic discharges, generalized periodic discharges, lateralized rhythmic delta activity, generalized rhythmic delta activity, and other patterns. Subsequently, eight clinicians without specialized EEG or machine-learning expertise were invited to classify 100 EEG samples into one of the six above categories. Participant selection reflected a real-world user cohort without pre-requisite knowledge. The participants were separated into two groups randomly, one given AI assistance in stage 1 and the other in stage 2, with the two stages being two weeks apart. All were asked to complete a post-study survey. The mean diagnostic accuracy with ProtoPMed-EEG was 71% vs 47% without the AI model (p < 0.05). The superior performance with AI was observed in all clinicians. With AI assistance, the mean rater-to-rater interrater reliability percent agreement was 62% compared to 47% without. However, the average time needed to diagnose was longer with AI use (32±33 seconds with AI vs 25±39 seconds without). In the post-study survey, 7/8 clinicians recommended the AI model for educating future medical professionals. In conclusion, the authors assessed ProtoPMed-EEG’s ability to assist clinicians with EEG diagnoses. Its success in this study demonstrated its potential to become a teaching tool for medical trainees and an aid in clinical situations.
Image: PD
©2024 2 Minute Medicine, Inc. All rights reserved. No works may be reproduced without expressed written consent from 2 Minute Medicine, Inc. Inquire about licensing here. No article should be construed as medical advice and is not intended as such by the authors or by 2 Minute Medicine, Inc.
