Gamified or Glorified? A systematic review of serious games for security & privacy in the SDLC

Abstract

Context:

While security and privacy are playing increasingly important roles in the software development process, the skill shortage for security and privacy keeps growing. To address this, academia and industry alike have proposed game-based approaches, to foster involvement of non-expert stakeholders, and to improve collaboration among the various parties involved. However, research has shown that injudicious implementation of gamification can do more harm than good. Basing the game design on an existing, established methodology is crucial to accomplish the intended goals of the game.

Objective:

This paper identifies and compares the different serious games in the space of security and privacy engineering. It highlights the differences between games in goals, intent, form, and approach, and pays particular attention to (1) the specific motivations behind the selected gameful design elements, and (2) the scientific evidence of the benefits of these game-based approaches.

Method:

We perform a widely-scoped discovery search for relevant serious games to establish a dataset of relevant games. For each of the in total twelve games, we collect and study the different game artifacts, covered CyBOK knowledge domains, materials, research articles, and practitioner testimonials.

Results:

Most games target a multi-stakeholder industry practitioner audience, typically with the goal of providing a first introduction to activities such as Requirements Engineering and Threat Modeling. The majority of games have been designed in an ad-hoc manner, rather than being based on design frameworks or methodologies. Scientific evaluations of these games mostly focus on obtaining participant feedback, experiences and opinions, rather than evaluating the actual outcomes of applying the game.

Conclusions:

While game-based approaches for security and privacy in the SDLC are showing promise, many of them have not been designed with proven serious game design frameworks or methodologies. Further empirical evidence is required to confirm the effectiveness of these games.