A Home Hemodialysis Objective Structured Clinical Examination (OSCE) for Formative Assessment of Nephrology Fellows.

Abstract

BACKGROUND The Accreditation Council for Graduate Medical Education requires graduating nephrology fellows demonstrate competence in home hemodialysis (HHD). Because low patient numbers may lead to training gaps, clinical experience may be enhanced using simulation. We designed and preliminarily validated a formative objective structured clinical examination (OSCE) assessing clinical care of an uncomplicated patient initiating HHD, using a unified model of construct validity. METHODS The OSCE was developed by a nephrology fellow and five faculty (three practicing HHD). The nine-member test committee (five in HHD practice; one HHD care partner) assessed test item difficulty/relevance and determined passing scores. The final test consisted of 27 items (31 possible points); seven were evidence-based/standard-of-care questions (9.5 possible points). Passing score was 20 out of 31 points (65%). Median relevance for all items was "important" or "essential". Content validity index was 0.84. On preliminary validation by 11 board-certified volunteers (four practicing HHD), overall mean±SD score was 27.5±2 (100% passing); kappa=0.83 [95% confidence interval (CI) 0.67-0.99]. Validator evidence-based question score was 9.0 ± 0.6 . RESULTS Thirty-eight fellows (nine programs, 21 first-year; 17 second-year) were tested. Seventy one percent passed (Cronbach's alpha=0.70). Fellows' mean±SD scores were lower than validators: 21.5±4.0 vs. 27.5±2, p<0.001, as were their scores on evidence-based questions: 7.4±1.4 vs. 9.0±0.6, p<0.001. Eighty-eight percent of evidence-based/standard-of-care questions were answered correctly by validators vs. 62% by fellows; p <0.001. Forty-two percent of fellows were able to name four potential benefits and two risks associated with HHD; 79% recognized that the primary risk of buttonhole cannulation was infection. Seventy-four percent correctly identified minimum single pool Kt/V for thrice-weekly hemodialysis, and 29% knew the minimum standard weekly Kt/V target. Eighty-eight percent of fellows surveyed (22 out of 25) agreed/strongly agreed that the OSCE was useful in self-assessing proficiency. CONCLUSIONS The OSCE may be used as a formative assessment of fellow proficiency in prescribing HHD.