Statistical validation of rival models for observable stochastic process and its identification

Abstract

In this paper, for statistical validation of rival (analytical or simulation) models collected for modeling observable process in stochastic system (say, transportation or service system), a uniformly most powerful invariant (UMPI) test is developed from the generalized maximum likelihood ratio (GMLR). This test can be considered as a result of a new approach to solving the Behrens-Fisher problem when covariance matrices of multivariate normal populations (compared with respect to their means) are different and unknown. The test makes use of an invariant statistic whose distribution, under the null hypothesis, does not depend on the unknown (nuisance) parameters. The sample size and threshold of the UMPI test are determined from minimization of the weighted sum of the model builder's risk and the model user's risk. The rules are proposed to identify an observable process with one of several rival models, suitable for modeling, which accurately represents the process, especially when decisions involving expensive resources are made on the basis of the results of the model. Application examples are given.