The effect of assessor expertise on reliability of analysis of video signs associated with concussion in Australian football.

Objectives: To determine whether spotters with medical training and experience in managing concussion have higher inter-rater reliability and accuracy than non-medical personnel when identifying video signs associated with concussion in Australian football.

Design: Retrospective cohort study.

Methods: Video clips were collected of all impacts potentially resulting in concussion during 2012 and 2013 Australian Football League (AFL) seasons. Raters were divided into medical doctors and a non-medical group comprising allied health practitioners (physiotherapists) and non-medical/non-allied health personnel (performance analysts). Raters assessed 102 randomly selected videos for signs of concussion. The inter-rater reliability was calculated. Sensitivity, specificity, positive and negative predictive values were calculated by comparing the rater responses to the consensus opinion from two highly experienced clinicians with expertise in concussion.

Results: No statistically significant difference in inter-rater reliability was observed between the medical doctors and the non-medical group. Both groups demonstrated good to excellent agreement for slow to get up, clutching at head/face and facial injury. Both groups displayed intra-class coefficient >0.55 for no protective action-floppy, loss of responsiveness, and motor incoordination, and displayed lowest agreement for no protective action-tonic posturing, impact seizure and blank/vacant look. No statistically significant difference was found between the groups for sensitivity, specificity, positive and negative predictive values for correctly classifying video signs compared to the expert consensus opinion.

Conclusions: After completing sufficient standardised training and testing, medical and non-medical personnel demonstrate comparable reliability in identifying video signs of concussion in professional Australian football and may be suitable for the role of video spotter.