Proposal of an alternative way of reporting the results of comparative simulation studies.

Abstract

Introduction: Monte Carlo simulation studies allow testing multiple experimental conditions, whose results are often difficult to communicate and visualize to their full extent. Some researchers have proposed alternatives to address this issue, highlighting its relevance. This article develops a new way of observing, analyzing, and presenting the results of simulation experiments and is explained step by step with an example.

Methods: A criterion is proposed to decide which results could be averaged and which results should not be averaged. It is also indicated how to construct Traceability Tables. These tables will show the behavior of the different analytical approaches studied under the chosen conditions and their variability under the averaged conditions. A way of observing the influence of the manipulated variables on the performance of the set of analysis approaches studied is also developed, Variability Set. Finally, a way of exposing the procedures that have the best performance in a particular condition is suggested.

Results and discussion: This Analysis Plan for reporting the results of simulation studies provides more information than existing alternative procedures, provides valuable information for method researchers, and specifies to applied researchers which statistic they should use in a particular condition. An R Shiny application is provided.