Supporting robust system analysis with the test matrix tool framework

Abstract

We present the Test Matrix Tool (TMT) framework, a simulation-agnostic framework providing end-to-end support for robust analysis of complex systems. The need to execute a large number of simulations is common to many problem environments, even those already reduced by Design of Experiments or similar methodologies. TMT addresses key end-user needs in easing the specification, execution and analysis of simulation workloads in ways that are consistent between specific applications of the framework. The TMT design contributes modular specifications for key data communicated between and within the specification, execution and analysis components. Our TMT implementation is an instantiation of those formats freely available for general use. TMT's data analysis component provides a variety of features data filtering, comparison, transformation and visualization for analytic tasks on any TMT-embedded model. We provide a brief case study as an example of its use in a real-world application.