An assessment of generative artificial intelligence in responding to clinical queries on tapering antidepressants.

Background: A substantial cohort of individuals rely on online resources, such as discussion forums, for support on tapering antidepressants. This study aimed to assess the performance of generative artificial intelligence (AI) in responding to clinical queries on tapering antidepressants.

Methods: Ten queries on tapering antidepressants were developed based on previous research, prescribing guidelines, and online peer support forums. Queries covered areas including reasons for discontinuing antidepressants, tapering methods, withdrawal symptoms, and relapse. Each query was submitted to ChatGPT (OpenAI, San Francisco, CA) using the GPT-4 model as an independent standalone query using standardised prompts. Responses were evaluated in terms of relevance, accuracy, completeness, and clarity by two researchers working independently.

Results: GPT-4 responses to all tapering queries were considered relevant and within scope. Most responses (8/10) incorporated safety netting by emphasising the importance of consulting healthcare professionals before making any medication changes. The overall accuracy, completeness, and clarity of responses compared less favourably. The response to a query on hyperbolic tapering had the least favourable assessment. This was due to inaccuracies as the response incorrectly referred to logarithmic reductions and provided inaccurate examples of fixed dosage reductions. Several instances of AI hallucinations were identified, including fabricated references.

Conclusion: Generative AI is having a transformative impact on healthcare, including how healthcare professionals and patients access information about clinical queries, such as antidepressant tapering. The study findings show that GPT-4 was able to provide relevant and safety-conscious responses on antidepressant tapering. However, performance issues such as inconsistencies and inaccuracies in tapering recommendations highlight the important role that healthcare professionals continue to play in providing patients with clinically trained, professional support in safely managing health-related issues. Further research on developing AI evaluation tools is needed to ensure consistency in the approaches used in evaluating the performance of AI in addressing clinical queries.