Validity evidence for games as assessment environments

Abstract

This study provides empirical evidence of a highly specific use of games in education— the assessment of the learner. Linear regressions were used to examine the predictive and convergent validity of a math game as assessment of mathematical understanding. Results indicate that prior knowledge significantly predicts game performance. Results also indicate that game performance significantly predicts posttest scores, even when controlling for prior knowledge. These results provide evidence that game performance taps into mathematical understanding. Introduction Games as assessment contexts Games have long been attractive as learning environments given that games can entertain, motivate, and energize us. This report will address a highly specific use of games in education—the assessment of the learner. Games can be used as formative (in-the-processof-learning) assessments, as well as for criterion trials, either to determine the level of performance of an individual or to gauge the speed and agility with which a learner acquires a new set of skills in an unfamiliar game environment (Baker & Delacruz, 2007; Gee, 2008). When designed properly, the underlying game engine can enable increases in challenge, complexity, and the cognitive demands required as the game progresses such that game play can be one form of assessment. Assessment is a process of drawing reasonable inferences about what a person knows by evaluating what they say or do in a given situation. However, it is insufficient to state that an assessment task is or is not valid. Rather, determining the validity of assessment tasks requires creating an argument that examines how well assessments answer the questions they purport to answer, as well as ensuring the data obtained provide the appropriate evidential basis for the claims made about students (American Educational Research Association, American Psychological Association, and National Council for Measurement in Education, Standards for Educational and Psychological Testing, 1999). In this study, we report findings that investigated the validity of a mathematics game as assessment of mathematical understanding by examining the relationship between mathematical knowledge and performance in the game.