Appropriate semantic qualifiers increase diagnostic accuracy when using a clinical decision support system: a randomized controlled trial.

Abstract

Background: The role of appropriate semantic qualifiers (SQs) in the effective use of a clinical decision support system (CDSS) is not yet fully understood. Previous studies have not investigated the input. This study aimed to investigate whether the appropriateness of SQs modified the impact of CDSS on diagnostic accuracy among medical students.

Methods: For this randomized controlled trial, a total of forty-two fifth-year medical students in a clinical clerkship at Chiba University Hospital were enrolled from May to December 2020. They were divided into the CDSS (CDSS use; 22 participants) and control groups (no CDSS use; 20 participants). Students were presented with ten expert-developed case vignettes asking for SQs and a diagnosis. Three appropriate SQs were established for each case vignette. The participants were awarded one point for each SQ that was consistent with the set SQs. Those with two or more points were considered to have provided appropriate SQs. The CDSS used was the Current Decision Support^Ⓡ. We evaluated diagnostic accuracy and the appropriateness of SQ differences between the CDSS and control groups.

Results: Data from all 42 participants were analyzed. The CDSS and control groups provided 133 (60.5%; 220 answers) and 115 (57.5%; 200 answers) appropriate SQs, respectively. Among CDSS users, diagnostic accuracy was significantly higher with appropriate SQs compared to inappropriate SQs (χ²(1) = 4.97, p = 0.026). With appropriate SQs, diagnostic accuracy was significantly higher in the CDSS group compared to the control group (χ²(1) = 1.16 × 10, p < 0.001). With inappropriate SQs, there was no significant difference in diagnostic accuracy between the two groups (χ²(1) = 8.62 × 10^-2, p = 0.769).

Conclusions: Medical students may make more accurate diagnoses using the CDSS if appropriate SQs are set. Improving students' ability to set appropriate SQs may improve the effectiveness of CDSS use.

Trial registration: This study was registered with the University Hospital Medical Information Network Clinical Trials Registry on 24/12/2020 (Unique trial number: UMIN000042831).