Inter-rater Agreement of Richmond Agitation Sedation Scale Assessments in Adult Patients Receiving Mechanical Ventilation in the ICU: A Cross-Sectional Study.

Importance: Accurate and reliable sedation assessment is crucial to improving patient outcomes in the ICU.

Objective: To evaluate the inter-rater agreement and reliability of Richmond Agitation Sedation Scale (RASS) assessments between bedside nurses and trained investigators in patients receiving mechanical ventilation in the ICU.

Design, Setting, And Participants: An assessor triad, comprising an ICU nurse providing direct patient care and two trained investigators, simultaneously performed RASS assessments during 79 encounters with 62 unique patients receiving mechanical ventilation at two ICUs at a tertiary care academic hospital in Colorado. A total of 58 nurses participated in the study.

Main Outcomes And Measures: The inter-rater reliability of RASS assessments was evaluated with the intraclass correlation coefficient (ICC) and weighted kappa (κ), and inter-rater agreement was evaluated with percentage agreement and Bland-Altman analysis.

Results: Acute respiratory failure (55%) and altered mental status (21%) were the most common reasons for mechanical ventilation. Most patients were receiving one (interquartile range, 0.5-2) continuous sedative during the assessment. The inter-rater reliability of RASS assessments between the nurses and investigators (ICC, 0.728-0.779; weighted κ, 0.62-0.63) was lower than between the two investigators (ICC, 0.891; weighted κ, 0.80). The assessor triad agreed on the same RASS values in only 35% of observations. The average differences in RASS were greater between the investigators and nurses, ranging from -0.658 to -0.544, compared with 0.114 between the two investigators. Compared with the mean of the two investigators, RASS values recorded by nurses were more likely to be higher (52% of observations), indicating a lighter sedation level. In 16% of observations, at least one assessor commented on uncertainty or ambiguity with the RASS.

Conclusions And Relevance: The inter-rater reliability of RASS assessments was high. However, we observed variations in the degree of agreement by assessor category. Further studies are necessary to explore how factors such as assessor characteristics, ICU environment, and patient conditions influence the inter-rater agreement of the RASS in contemporary ICU practices.