Development and Validation of a Shave Biopsy Training Checklist.

Abstract

Background and objectives: Residencies train residents in procedures and assess their competency, but existing assessment tools have demonstrated poor reliability and have not been validated.

Methods: This mixed-methods study validated a shave biopsy checklist with family medicine and dermatology faculty at two academic centers. In each phase of the study, teaching faculty scored a video-recorded simulated procedure using the checklist, and investigators assessed content validity, interrater reliability, and accuracy.

Results: In focus groups of nine family medicine and dermatology faculty, 16 of 18 checklist items met or surpassed 80% interrater reliability. Overall checklist reliability was 74%. Focus group surveys initially revealed insufficient content validity. Lowest performing items were removed, and then the follow-up content validity index (0.76) surpassed the required threshold (0.62). Twenty-one of 70 family medicine faculty completed a final survey, which showed a content validity index of 0.63, surpassing the required threshold of 0.42. Twelve of 70 family medicine faculty viewed and scored a simulated video-recorded procedure. Overall interrater reliability was 91% (Cohen's d=1.36). Fourteen of 16 checklist items demonstrated greater than or equal to 90% interrater reliability. Accuracy analysis revealed 67.9% correct responses in focus groups and 84.9% in final testing (simple t test, P<.001, Cohen's d=1.4).

Conclusions: This rigorously validated checklist demonstrates appropriate content validity, interrater reliability, and accuracy. Findings support use of this shave biopsy checklist as an objective mastery standard for medical education and as a tool for formative assessment of procedural competency.