Photo Credit: Amir Sajjad
The following is a summary of “Recommended antibiotic treatment agreement between infectious diseases specialists and ChatGPT®,” published in the January 2025 issue of Infectious Disease by Montiel-Romero et al.
Antimicrobial resistance poses a global public health threat, and Chat Generative Pre-Trained Transformer (ChatGPT®) could analyze antimicrobial susceptibility test data in real-time, particularly in areas lacking infectious disease (ID) specialists.
Researchers conducted a retrospective study to evaluate the agreement between ChatGPT® and ID specialists on appropriate antibiotic prescriptions in simulated cases.
They used data from microbiological isolates at the center to create 100 patient cases with various infections, each including age, infectious syndrome, isolated organism, and complete antibiogram. Following a set of instructions, the cases were input into ChatGPT® and presented to 5 ID specialists. For each case, they 2 questions: “What is the most appropriate antibiotic for this patient?” and “What is the most probable mechanism of resistance based on the antibiogram?” Agreement between ID specialists and ChatGPT® was calculated, along with Cohen’s kappa coefficient.
The results showed the agreement on the recommended antibiotic prescription between ID specialists and ChatGPT® occurred in 51/100 cases, with a kappa coefficient of 0.48. An agreement on antimicrobial resistance mechanisms was found in 42/100 cases, with a kappa coefficient of 0.39. In the subanalysis by infectious syndromes and microorganisms, agreement ranged from 25% to 80%, and kappa coefficients varied from 0.21 to 0.79.
Investigators concluded the poor agreement between ID specialists and ChatGPT® in determining the optimal antibiotic management strategies for simulated clinical case scenarios.
Source: bmcinfectdis.biomedcentral.com/articles/10.1186/s12879-024-10426-9
