2024 Scholars' Research Symposium Abstract: Metacognitive Analysis of Simulation-Based Education Identifies a Gap in Healthcare Education.

Introduction: Simulation has become an integral part of healthcare education. Studies demonstrate rapid knowledge and skill acquisition with the use of simulation and rapid knowledge degradation if it is not further reinforced. Effect of simulation on metacognitive processes, or the ability to understand one's own knowledge, is not well-investigated yet. The University of South Dakota Sanford School of Medicine in the U.S. and Royal College of Surgeons in Ireland - Bahrain campus, collaborated on combining the best simulation-based and team-based learning (TBL) practices to investigate team dynamics and knowledge exchange among team members in high-fidelity simulation (HFS) for 3rd year medical students. The purpose of the study was to assess metacognitive processes and their impact by simulation.

Methods: The activity consisted of four clinical scenarios (atrial fibrillation, anaphylaxis, meningitis complicated with septic shock and ARDS, and burns). A total of 68 students were tested before the activity for their concrete knowledge and its level of assurance. Immediately after the activity and prior to debriefing, students were assessed for their conceptual knowledge relevant for the simulation scenario and its level of assurance. Knowledge test answers had values of "+1" and "-1" for correct and incorrect answers respectively, while the assurance level was assessed on a scale from 1 to 4. Calibration index (CI) was calculated as a product of knowledge-based answer value and level of assurance, thus its value ranged from "-4" to "+4."

Results: Data analysis revealed a weak correlation between individual-readiness assurance test (I-RAT) and insimulation decisions (Spearman correlation, 0.336, p less than 0.001). Performance in I-RAT showed non-significant correlation with either the decision making in the simulation or post-simulation test. Irrespective of accuracy, students maintained the same level of certainty across all the assessments (I-RAT, in-simulation, and post-simulation) with Spearman correlation Rho: I-RAT vs in-simulation (case 1) = 0.354, p less than 0.005, I-RAT vs in-simulation (case 2) = 0.419 p less than 0.001, and in-simulation (case 1) vs in-simulation (case 2) = 0.505, p less than 0.001.

Conclusions: The obtained results demonstrated low correlation between knowledge correctness and level of certainty. Additionally, learners demonstrated misinformed confidence. This was true for both extremes: some students were overconfident in their wrong answers while others struggled to be assured in their correct answers. This indicated that current simulation educational curricula do not enhance skills in self-assessment. Furthermore, multiple studies have shown medical students lack this skill, highlighting the prevalence of both imposter syndrome and cognitive bias contributing to overconfidence. These are concerning findings considering healthcare requires its providers to be involved in lifelong learning. Efficient, reliable learning with a high level of accuracy is unlikely to occur without a correct knowledge of self-appraisal. Formal healthcare curricula should address this need.