Evaluation of Econometric Models of Adaptive Learning by Predictive Measures

Abstract

Game-theoretic models of learning are hard to study even in the laboratory setting due to econometric and practical concerns (like the limited length of an experimental session).

In particular, as the simulations by (Salmon, 2001) show, in a cross-model (or "blind'') testing of several models, the data generated by those models does not correspond to the estimated parameters correctly.

Thus, even when the real data generation process is known we cannot distinguish correct models from incorrect ones by looking at the estimates.

However, we demonstrate that under the same conditions, models are clearly distinguishable if we compare predictions that the models make instead of comparing the model parameters.

We also provide a rationale for why this cross-model predictive quality is a particularly relevant way for improving learning models.