Efficient Learning of Finite Mixture Densities Using Mutual Information

This paper presents a technique of determining the optimum number of components in a mixture model. A count of the number of local maxima in the density of the data is first used to obtain a rough guess of the actual number of components. Mutual Information criteria are then used to judge if components need to be added or removed in order to reach the optimum number. An incremental K-means algorithm is used to add components to the mixture model if required. An obvious advantage of the proposed method is in terms of computational time, as a good guess of the optimum number of components is quickly obtained. The technique has been successfully tested on a variety of univariate as well as bivariate simulated data and the Iris dataset.