A construct validity analysis of the military general surgeon Knowledge, Skills, and Abilities methodology with respect to procedural complexity.

Background: The Knowledge, Skills, and Ability-Clinical Activity (KSA-CA) methodology was developed to gauge surgeon readiness; it uses Current Procedural Terminology codes organized into high- and low-acuity procedural groups. This study will evaluate if the methodology has a measure of procedural complexity.

Methods: Deidentified case logs from 41 general surgery residents (years 1-5) were analyzed. Individual Current Procedural Terminology codes were converted into KSA-CA scores and work relative value units (wRVUs). The wRVUs was used as surrogate measure of complexity. Correlations between the KSA-CA scores and median wRVU values, and the relationship between these metrics with the residents' clinical year were examined with Spearmen rank correlations ( ρ ).

Results: The KSA-CA scores and wRVU values were strongly correlated with the training year ( ρ > 0.85, p < 0.0001); the case logs were also correctly ranked by the KSA-CA score. The KSA-CA points and median wRVU values for the high-acuity procedures were also strongly correlated ( ρ = 0.76, p = 0.0001). Finally, the wRVU value variability within most procedural groups was noted to be large; analysis of this variability identified areas where basic surgical skills were overrepresented in the KSA-CA score.

Conclusion: The KSA-CA methodology is a valid measure of residents' case logs by clinical year. Also, the residents' KSA-CA scores and their median wRVU values were strongly correlated. These results support the validity of the KSA score as a proxy to procedural complexity. Several suggestions for the refinement of the KSA methodology were also presented to address overrepresentation of some basic clinical skills in the KSA-CA methodology.

Level Of Evidence: Diagnostic Test or Criteria; Level IV.