Scientists from the University of Maryland found that the responses generated by the chatbot ChatGPT in 88% of cases contained accurate information about breast cancer screening. The results of the research were published in the journal Radiology.
The study authors formulated 25 questions regarding recommendations for breast cancer screening. They submitted each question to ChatGPT three times (chatbot has been known to change its answer every time a question is asked.)
Three radiologists trained in mammography evaluated the responses. They found that the answers were appropriate for 22 out of 25 questions. ChatGPT was right about questions about breast cancer symptoms, risk factors, and recommendations for mammogram frequency and cost. Scientists noted that the information is presented in an understandable and accessible form.
One response was based on outdated information: ChatGPT suggested that mammograms be delayed for four to six weeks after COVID-19 vaccination. The February 2022 recommendations do not recommend postponing the procedure.
The other two questions were given inconsistent answers that varied significantly each time.
The team is now examining ChatGPT’s ability to deliver lung cancer screening guidelines and is also looking for ways to improve its guidelines. The scientists added that the medical community has a responsibility to evaluate these technologies and protect patients from potential harm from inappropriate screening recommendations.