A multivariate rank sum test for network simulation validation

Abstract

A primary goal of any verification and validation process should be to enhance both the correctness of a simulation and the confidence placed in its results. One challenge is to develop a process that is at the same time feasible and compatible with an organization's needs, and can be applied to both existing simulations as well as new ones. Multivariate methods can be used to test the hypothesis of agreement between simulated predictions and empirical observations. This paper describes a statistical test useful for the validation of simulations of battlefield communications networks. The method employs a multivariate nonparametric rank sum test with the aid of a computer-intensive randomization procedure to assess the significance of the defined test statistic. For illustrative purposes, the validation procedure is applied to a simulation that was developed to duplicate a configuration tested, in which "messages" were passed over a communications network using the combination of the Tactical Fire Direction System (TACFIRE) protocol and Single-Channel Ground and Airborne Radio System (SINCGARS) Combat Net Radios (CNR). The purpose of this research is to strengthen the link between experimentation and simulation, both of which should be utilized in evaluating communications systems' measures of performance (MOP).<>