Exploiting Prior Information in Stochastic Knowledge Assessment

Abstract

Various adaptive procedures for efficiently assessing the knowledge state of an individual have been developed within the theory of knowledge structures. These procedures set out to draw a detailed picture of an individual’s knowledge in a certain field by posing a minimal number of questions. While research so far mostly emphasized theoretical issues, the present paper focuses on an empirical evaluation of probabilistic assessment. It reports on simulation data showing that both efficiency and accuracy of the assessment exhibit considerable sensitivity to the choice of parameters and prior information as captured by the initial likelihood of the knowledge states. In order to deal with problems that arise from incorrect prior information, an extension of the probabilistic assessment is proposed. Systematic simulations provide evidence for the efficiency and robustness of the proposed extension, as well as its feasibility in terms of computational costs.