Musculoskeletal infection reporting and data system (MSKI-RADS): multi-institutional wider generalizability study.

Objective: To determine inter-reader reliability and diagnostic accuracy of MSKI-RADS among early-career radiologists with different experience levels.

Materials And Methods: This is a retrospective multicenter study of MRIs of proven musculoskeletal (MSK) infections. MSKI-RADS categories included: 0-incomplete imaging, I-negative for infection, II-superficial soft tissue infection, III-deeper soft tissue infection, IV-possible osteomyelitis (OM), V-highly suggestive of OM, VI-known OM, and NOS-nonspecific bony lesions unrelated to infection. There were 22 readers from 9 institutions with 1-8 years of radiology experience, including radiology residents, MSK radiology clinical fellows, and junior MSK attendings. After initial training with expert readers, a dataset of 210 cases across a spectrum of extremity infections from a single institution was evaluated. The readers recorded the following for each case: MSKI-RADS score, final qualitative diagnosis, and confidence levels for both. Inter-reader agreements (ICC) and accuracy were obtained.

Results: Among 210 cases, there were 17 negative, 32 superficial soft tissue infections, 39 deeper soft tissue infections, 24 possible OM, 41 highly suggestive of OM, 18 known OM, and 39 NOS cases. A moderate inter-reader agreement was seen for 22 readers (ICC: 0.57 (CI = 0.52, 0.67)). The average true positive rates for non-infectious cases (I and NOS), soft tissue infection (II and III), and bony infection (IV and V) were 70% (95% CI: 67-73%), 66% (95% CI: 63-68%), and 82% (95% CI: 80-84%), respectively. Average true positive rates for classes V and VI were 80% (95% CI: 77-82%) and 86% (95% CI: 82-89%), respectively. Overall, MSKI-RADS reader accuracy was 66 ± 10%, higher than the qualitative diagnosis accuracy of 59 ± 11% (p < 0.05). Average confidence levels for MSKI-RADS score and final qualitative diagnosis were 3.56 ± 0.91 and 3.52 ± 0.92, respectively (p > 0.05).

Conclusion: MSKI-RADS is an accurate and reliable MRI-based classification system for the diagnosis of a spectrum of musculoskeletal infections among early-career radiologists, like expert readers.

Key Points: Question What is the inter-reader reliability and diagnostic accuracy of the musculoskeletal infection reporting and data system (MSKI-RADS) among early-career radiologists with different experience levels? Findings Overall reader accuracy using MSKI-RADS was significantly higher than the accuracy for final qualitative diagnoses, like the results of the initial validation study with expert readers. Clinical relevance MSKI-RADS is valid and reliable for diagnosis and grading of a spectrum of extremity musculoskeletal infections for early career radiologists of different experience levels, like expert readers.