Pilot Study: Development and Evaluation of Validity Evidence of a Low-Cost, Low-Fidelity Hand Model to Teach Clinical Assessment of Small Joint Swelling in Inflammatory Arthritis.

Abstract

Objective: To create an affordable, easily reproducible, low-fidelity hand model and present a validity argument for its use to support residents' learning to identify inflammatory arthritis.

Methods: We designed a hand model to simulate small joint swelling and evaluated evidence to support its use using Messick's Framework between April 2023 and April 2024. Rheumatologists nationwide rated our model (content validity). At 2 tertiary institutions, rheumatologists (experts) and internal medicine residents (learners) evaluated 3 sets of models for small joint swelling, and their scores were compared with Mann-Whitney U test (construct validity). Learners applied their skills to 3 patient encounters, and patient versus model scores were compared with Wilcoxon signed rank test (predictive validity).

Results: One set of 2 hand models cost less than US $5 to create. Thirteen rheumatologists rated our model at 7.23 ± 2.52 out of 10 points; 11 of the 13 (84.6%) rheumatologists thought the model was helpful for training learners to examine swollen joints. Median model evaluation scores for 12 experts (98.9%; range, 92.2%-100%) and 32 learners (100%; range, 84.4%-100%) were not significantly different (p = 0.143). The 18 learners who also evaluated patients had a significantly lower median score when evaluating patients versus the hand models (difference 10.0%; range, 5.6-15.6; p < 0.001).

Conclusion: Content validity evidence supported our model's use in internal medicine training; however, the model needs improvement for greater construct and predictive validity. Our easily constructed, low-cost hand model may be a promising introductory training tool for rheumatology medical education.