The Orbital Grading system yields higher precision than the Matsushima grading system in assessing angiographic outcomes after EDAS for Moyamoya disease: an interrater reliability analysis.

Matsushima Grading System has been widely used to evaluate vessel ingrowth after bypass procedures for patients with Moyamoya disease (MMD). However, Matsushima is notably subjective and prone to measurement variability among clinicians. To address these limitations, the Orbital Grading System was developed as an objective approach to assess collateralization after EDAS, utilizing orbital craniometric landmarks to segment the ipsilateral skull into three distinct regions. The current study compares the interrater reliability of both scales. Six physicians, including two attending hybrid-neurovascular surgeons, one interventional neurologist, one neurointerventional radiologist and two vascular neurosurgery fellows, independently graded digital subtraction angiography (DSA) images from 37 hemispheres of MMD-patients using both the Matsushima and Orbital Grading scales. Fleiss'-kappa was used to assess inter-rater agreement for each scale, with results interpreted according to Landis-and-Koch-agreement-scale. All six raters graded 37 DSA-images using both Matsushima and the Orbital grading systems. Both scales demonstrated statistically significant inter-rater agreement (p < 0.0001). However, the Orbital Grading System produced significantly higher agreement (Kappa:0.83 [95%CI:0.74-0.93], indicating almost-perfect-agreement) compared to Matsushima grading system (Kappa:0.52 [95%CI:0.38-0.65], indicating moderate-agreement). Simple percentage agreement among all raters was 81.1% in the Orbital Grading System and 48.6% in Matsushima Grading. The Orbital Grading System demonstrates superior interrater reliability compared to the Matsushima grading, with significantly higher agreement among raters. This suggests that the Orbital Grading System is a more consistently objective tool for evaluating collateralization following EDAS. By reducing grading discrepancies between observers and enhancing generalizability, this system has the potential to optimize the management of MMD. Although our study demonstrates excellent reproducibility of the Orbital Grading System, future studies are needed to validate its accuracy in reflecting the true extent and functional significance of postoperative collateral formation.