Asymptotically optimal function estimation by minimum complexity criteria

Abstract

The minimum description length principle applied to function estimation can yield a criterion of the form log(likelihood)+const/spl middot/m instead of the familiar log(likelihood)+(m/2) log n where m is the number of parameters and n is the sample size. The improved criterion yields minimax optimal rates for redundancy and statistical risk. The analysis suggests an information-theoretic reconciliation of criteria proposed by Rissanen (1983), Schwarz (1978), and Akaike (1973).<>