Using game analytics to evaluate puzzle design and level progression in a serious game

Our previous work has demonstrated that players who perceive a game as more challenging are likely to perceive greater learning from that game [8]. However, this may not be the case for all sources of challenge. In this study of a Science learning game called Quantum Spectre, we found that students' progress through the first zone of the game seemed to encounter a "roadblock" during gameplay, dropping out when they cannot (or do not want to) progress further. Previously we had identified two primary types of errors in the learning game, Quantum Spectre: Science Errors related to the game's core educational content; and Puzzle Errors related to rules of the game but not to science knowledge. Using this prior analysis, alongside Survival Analysis techniques for analyzing time-series data and drop-out rates, we explored players' gameplay patterns to help us understand player dropout in Quantum Spectre. These results demonstrate that modeling player behavior can be useful for both assessing learning and for designing complex problem solving content for learning environments.