Can ChatGPT answer patients' questions about nuclear stress tests and F-Flurodeoxyglucose PET for myocardial inflammation?

Background: Several modalities are used for stress testing and require specific patient preparation. F-Flurodeoxyglucose positron emission tomography (FDG PET) is an important tool in the diagnosis and risk stratification of patients suspected of cardiac sarcoidosis and endocarditis [1-3]. There is a need for improved patient access to questions regarding cardiac testing to ensure proper adherence to instructions. We sought to evaluate the effectiveness of ChatGPT in answering questions about stress testing and cardiac FDG PET inflammation scans.

Methods And Results: We generated fifty-eight questions about stress testing and cardiac FDG PET inflammation scans. OpenAI ChatGPT-3.5 and -4o were used to answer the questions. The answers were graded by three nuclear cardiologists as the following categories: 1 = correct and complete, 2 = somewhat correct/somewhat complete, 3 = incorrect: no benefit, or 4 = incorrect: harmful/misleading information (Table I). Of the 174 grades assigned to responses to the questions from ChatGPT-3.5, 62/174 (36%) were correct and complete, 93/174 (53%) were somewhat correct/somewhat complete, 12/174 (7%) were incorrect: no benefit, 7/174 (4%) were incorrect: harmful/misleading information. Of the grades assigned to responses to questions from ChatGPT-4o, 107/174 (61%) were correct and complete, 62/174 (36%) were somewhat correct/somewhat complete, 3/174 (2%) were incorrect: no benefit, and 2/174 (1%) were incorrect: harmful/misleading information.

Conclusions: ChatGPT can provide some accurate responses to patient questions regarding stress tests and cardiac FDG PET inflammation studies, and its accuracy has improved over time; however, it is not suitable as a primary resource for clinical care at this stage of development.