Fisher information, stochastic complexity, and universal modeling

The main objective in universal modeling is to construct a process for a class of model processes which for long strings, generated by any of the models in the class, behaves like the data generating one. Hence, such a universal process may be taken as a representation of the entire model class to be used for statistical inference. If f(x/sup n/) denotes the probability or density it assigns to the data string x/sup n/=x/sub 1/,..,x/sub n/, then the negative logarithm - log f(x/sup n/), which may be viewed as the shortest ideal code length for the data obtainable with the model class, is called the stochastic complexity of the string, relative to the considered model class. Unlike in related universal modeling, where the mean code length is sufficient, we also need an explicit asymptotic formula for the stochastic complexity. This is because it permits a comparison of different model classes by their stochastic complexity in accordance with the MDL (minimum description length) principle.