Visualizing cognitive processes in medical education: Forward and backward reasoning in a digital family medicine assessment.

Abstract

Background: When evaluating medical students, wouldn't it be helpful to visualize and understand your students' clinical reasoning (CR)? There are different types of CR for students to use in their multiple-choice questions (MCQs). While forward reasoning uses data to generate a hypothesis, backward reasoning looks at possible prompts (answers) to generate a hypothesis. This study implements a new approach to visualizing CR during digital MCQ assessment. Furthermore, it examines the effect of feedback given during the learning process, also known as formative feedback, on the thought process.

Methods: Quantitative and qualitative data were collected at the end of two consecutive year 5 end-of-semester Family Medicine exams in 2023. Both exams consisted of 60 MCQs and an additional research component also comprising MCQs. During the research component students digitally recorded their reasoning process while answering MCQs. Qualitative data were coded via three rounds of coding, including two marking/coding parties.

Results: This study was able to digitally visualize CR in a large cohort (n=210). On average, the exam questions were answered with 87% of CR. Forward reasoning was used significantly more often than backward reasoning (WS 22/23 p=0.006, SS 23 p<0.001). High performers were significantly more likely to use forward reasoning and backward reasoning than low performers (p<0.01). Formative Feedback had no significant influence on the choice of CR type (p=0.281). Follow-up questions might stimulate a change in CR behaviour; however further research is needed (p<0.001).

Conclusion: This study illustrates an alternative method to visualize students' cognitive processes on a large scale. This approach sheds light on the required cognitive processes. It may help educators to better understand what to focus on during curricular learning activities aimed at preparing for state exams. This method may be beneficial as a quality criterion for MCQ questions, as it does not only rely on expert opinion or question metrics but illustrates students' cognitive processes when answering MCQ.