A small sample model selection criterion based on Kullback's symmetric divergence

Abstract

The Kullback information criterion (KIC) is a recently developed tool for statistical model selection (Cavanaugh, J.E., Statistics and Probability Letters, vol.42, p.333-43, 1999). KIC serves as an asymptotically unbiased estimator of a variant of the Kullback symmetric divergence, known also as J-divergence. A bias correction of the Kullback symmetric information criterion is derived for linear models. The correction is of particular use when the sample size is small or when the number of fitted parameters is of a moderate to large fraction of the sample size. For linear regression models, the corrected method, called KICc, is an exactly unbiased estimator of a variant of the Kullback symmetric divergence between the true unknown model and the candidate fitted model. Furthermore, KICc is found to provide better model order choice than any other asymptotically efficient methods when applied to autoregressive time series models.