Assessment of a Deep Learning Model Trained on Permanent Pathology for the Classification of Squamous Cell Carcinoma in Mohs Frozen Sections: Lessons Learned.

Background: There is a scarcity of artificial intelligence models trained on frozen pathology. One way to expand the clinical utility of models trained on permanent pathology is by applying them to frozen sections and fine-tune based on weaknesses.

Objective: To qualitatively evaluate a deep learning model trained on permanent pathology to classify squamous cell carcinoma on Mohs surgery frozen sections to learn model shortcomings and inform retraining and fine-tuning.

Materials And Methods: The authors trained a model for classification of tumor on 746 skin biopsy slides and tested it on 15 Mohs surgery frozen sections. The authors estimated performance metrics and compared the regions of interest generated by the model with the original H&E slides.

Results: The model achieved an AUC-ROC of 0.985 and 0.796 for tumor classification in permanent pathology and in frozen sections, respectively. Regions of interest for frozen sections with scarce tumor areas were inaccurate, focusing on normal tissue for slides classified as false negative, or highlighting structures different from tumor (e.g., inflammation, muscle, and nerves) for slides classified as true positive.

Conclusion: Deep anatomical structures more commonly present in Mohs frozen pathology might represent data out-of-distribution for models trained on permanent pathology, potentially leading to inadequate model outputs.