Neural network for natural language processing to determine treatment urgency in an ophthalmology emergency department.

Background: In an ophthalmology emergency department, determining treatment urgency is crucial for patient safety and the efficient use of resources. The aim of this study was to use artificial intelligence to develop a neural network and evaluate its accuracy in predicting treatment urgency.

Methods: In a retrospective study, a medical history questionnaire consisting of a free-text section and checkbox questions was given to 1715 patients on arrival and the responses were used as input data. The data were labelled as either emergency or non-urgent according to the patients' diagnosis and divided into a training and a test dataset. A bidirectional encoder representations from transformers (BERT) neural network for language processing was concatenated with a multilayer perceptron (MLP), and the result was processed by another MLP. Performance metrics were determined on the test dataset. Particular emphasis was placed on explainable artificial intelligence methods.

Results: The combination of BERT and MLP achieved the highest F1 score of 0.81, with sensitivity of 0.89 and specificity of 0.58. The three ophthalmologists achieved F1 scores of 0.77, 0.75 and 0.73, respectively. Most of the input features that were identified by explainable artificial intelligence methods as being responsible for the network's decision correspond to clinically relevant features for the emergency/non-urgent decision.

Conclusions: A neural network was developed to identify treatment urgency based on natural language data and checkbox questions which could complement the initial assessment. Future studies could focus on validation and improving accuracy, as well as integrating the network into workflows and discussing ethical implications.