Photo Credit: FotografieLink

A significant proportion of AI responses to primary care-related questions are “accurate with missing information,” according to a study in the American Journal of Preventive Medicine. Joseph Kassab, MD, and colleagues assessed the capacity of ChatGPT-4 and Google Bard to deliver accurate recommendations in preventive medicine and primary care. The researchers presented 56 questions to ChatGPT-4 in June 2023 and Google Bard in October 2023, with two physicians independently reviewing responses. After reaching a consensus, 28.6% of ChatGPT-4-generated answers were accurate, 28.6% inaccurate, and 42.8% accurate with missing information. For Bard-generated answers, 53.6% were accurate, 17.8% inaccurate, and 28.6% accurate with missing information. Researchers found notable inaccuracies (80%) in both models in response to questions related to the CDC as well as immunization.