Empowering Early Career Neurosurgeons in the Critical Appraisal of Artificial Intelligence and Machine Learning: The Design and Evaluation of a Pilot Course.

Background: Artificial intelligence (AI) is expected to play a greater role in neurosurgery. There is a need for neurosurgeons capable of critically appraising AI literature to evaluate its implementation or communicate information to patients. However, there are a lack of courses delivered at a level appropriate for individuals to develop such skills. We assessed the impact of a 2-day (non-credit bearing) online digital literacy course on the ability of individuals to critically appraise AI literature in neurosurgery.

Methods: We performed a prospective, quasi-experimental non-randomized, controlled study with an intervention arm comprising individuals enrolled in our 2-day digital health literacy course and a waiting-list control arm used for comparison. We assessed participants' pre- and post-course knowledge, confidence, and course acceptability using Qualtrics surveys designed for the purpose of this study.

Results: A total of 62 individuals (33 participants, 29 waitlist controls), including neurosurgical trainees and both undergraduate and post-graduate students, attended the course and completed the pre-course survey. The 2 groups did not vary significantly in terms of age or demographics. Following the course, participants significantly improved in their knowledge of AI (mean difference = 3.86, 95% CI = 2.97-4.75, P-value < 0.0001) and confidence in critically appraising literature using AI (P-value = 0.002). Similar differences in knowledge (mean difference = 3.15, 95% CI = 1.82-4.47, P-value < 0.0001) and confidence (P-value < 0.0001) were found when compared to the control group.

Conclusions: Bespoke courses delivered at an appropriate level can improve clinicians' understanding of the application of AI in neurosurgery, without the need for in-depth technical knowledge or programming skills.