A systematic mapping study on serious game quality

Abstract

Context: A Serious Game (SG) is a game for purposes other than entertainment [12]. SGs are currently in widespread use and their popularity has begun to steadily increase; their application areas now extend not only to education, but also to military, health and corporate [9] [12] sectors. SGs are of vital importance at present, as they can be a means to achieve relevant goals from both a personal and an institutional point of view. This may take place in fields as diverse as defense, education, scientific exploration, health care, emergency management, city planning, engineering, religion, and politics. The number of users of these systems grows each day, signifying that their impact is very high, and it is precisely for this reason that more extensive research on SG quality is needed. Objective: The aim of this study is to discover the current state of SG quality initiatives, identifying gaps that merit future investigation. Method: We conducted a systematic mapping study (SMS) on SG quality, following the guidelines proposed by Kitchenham and Charters [7]. We selected 112 papers found in six digital libraries until April of 2013. Results: Since 2007, research on SG quality proves to have grown very significantly. Research has focused mostly on addressing the effectiveness of SGs (78.57%), in addition to several entertainment characteristics that are principally related to pleasure (62.50%). The most widely-researched software artifact was the final product (97.32%), with design coming very far behind (7.14%). Less than half of all the research reviewed had been validated by means of experiments, and in most of these cases, experiments were conducted by the same researchers who had proposed the SG. The majority of experiments have not been replicated. The most common research outcome was questionnaires, closely followed by the confirmation of knowledge. Most of these outcomes evaluated the quality of a particular SG. Conclusion: Results show that SG quality has undergone a very important growth, thus making SG quality an area of opportunity for future research. Researchers are mainly concerned with demonstrating or confirming the effectiveness of SGs, but very little research has been conducted as regards the characteristics of playability that make SGs more effective. Since effectiveness and playability are evaluated in the final product there is a need to provide quality assurance methods that incorporate quality issues from the early stages of SG development. Further empirical validation is also needed, and in particular, external replications must be performed in order to corroborate and generalize the findings obtained.