[The expert panel for Script Concordance Tests: A truly adequate reference?]

The Script Concordance Tests (SCTs) are an examination modality introduced by decree in the French National Ranking Exam for medical students in 2024. Their objective is to evaluate clinical reasoning in situations of uncertainty. In practice, SCTs assess the impact of new information on the probability of a hypothesis formulated a priori based on an authentic clinical scenario. This approach resembles probabilistic (or Bayesian) reasoning. Due to the uncertainty associated with the explored clinical situation, SCTs do not compare the student's response to an expected one in a theoretical knowledge reference. Instead, the distribution of responses from a panel of experienced physicians is used to establish the question's scoring scale. Literature data suggest that physicians, even experienced ones, like most humans, often exhibit biased intuitive probabilistic reasoning. These biases raise questions about the relevance of using expert panel responses as scoring scales for SCTs.