On Benchmarking and Validation in Wargames

There are multiple arguments for and against wargames. Many scientists do not recognise the science in wargames. It is suggested that there is not enough literature relating to wargaming, for there to be any large-scale research into wargames. This is primarily because scientists often refuse to publish results, thus creating a vicious cycle where research is not published because there is not enough research being published. This ultimately deters researchers from studying wargaming in any serious fashion. Owing to this limitation, published work on the results, and protocols of wargames are scarce in scholarly research. Wargaming has considerably less academic focus with a fragmented and practical focus on design and benchmarking. This is surprising given the long history of wargaming (dating back to the early 1600’s), when compared to the relatively recent history of other domains such as software engineering. To better understand the current state of research into wargaming in reference to benchmarking and validation, a scoping review (SR) was conducted. The scholarly research into wargaming reveals papers on general modelling, conflict modelling, influence modelling, evaluation of wargames, analytical tools, use of AI in wargame design, evaluation of predictive modelling in wargames, improving command and control through wargaming, and cost-benefit analysis for decision making. The initial analysis of the coverage of wargaming research, together with the limited number of papers found, indicate that there is a distinct lack of academic research into wargaming. Additionally, there is a wide variety of areas that are interested in the wargaming field, however, with no universal method of analysis or benchmarking, this limits the reproducibility of results, and the ability to judge the overall effectiveness of wargaming efforts. Wargame designers need to be able to assess wargame components to validate, compare, and predict the effects on gameplay and for decision-makers to draw conclusions with more confidence.