Length of the state trace: A method for partitioning model complexity

Abstract

A novel and easy-to-compute measure is proposed that compares the relative contribution of each parameter of a mathematical model to the model’s mathematical flexibility or complexity, with respect to accounting for the results of some specific experiment. When the data space is a two-dimensional plot of the type used in standard state-trace analysis, then the model complexity contributed by a single parameter equals the length of the state trace (LOST) that results when that parameter is varied and all other parameters are held constant. For the normal, equal-variance, signal-detection model, the average LOST when the response-criterion parameter $X_C$ is varied is about four times greater than the average LOST when the sensitivity parameter $d^{\prime }$ is varied. As a result, applying the signal-detection model to random data almost always leads to the conclusion that all the points share the same value of $d^{\prime }$ but were generated under different values of $X_C$. Parameters that have non-monotonic effects on performance, such as the attention-weight parameter that is used in popular exemplar and prototype models of categorization, tend to have large LOSTs, and therefore contribute to model flexibility more than parameters that have monotonic effects on performance. Comparing LOSTs for exemplar and prototype models also leads to some deep new insights into the structure of both models.