A new study suggests that AI-driven ChatGPT could match or even outperform physicians in addressing patients’ medical inquiries.
Researchers from the University of California, San Diego, Johns Hopkins, and other institutions put OpenAI’s ChatGPT to the test by posing 195 medical questions and comparing the chatbot’s responses to those of actual doctors on Reddit.
Healthcare experts, including internal medicine, pediatrics, oncology, and infectious disease specialists, assessed the chatbot and human responses on a five-point scale, considering the “quality of information” and “the compassion or bedside manner” exhibited.
In 78.6% of the 585 cases studied, medical professionals favored the chatbot’s answer over the physician’s. In addition, doctors’ responses scored 3.6 times higher in quality and 9.8 times higher in empathy than the chatbot’s answers.
ChatGPT’s responses tended to be more comprehensive than the doctors’, who often provided brief answers.
One key factor contributing to ChatGPT’s success in the study was its ability to offer more detailed and personalized answers compared to the physicians’ concise, time-saving responses.
For example, when asked about the risk of blindness after bleach exposure to the eye, ChatGPT empathized with the situation and gave a thorough explanation and clear instructions for eye-washing. In contrast, the doctor’s brief response offered only basic advice.
Despite its performance in this study, it is crucial to remember that ChatGPT is not a medical professional and should not be used for self-diagnosis.
