Clustering by multivariate mutual information under Chow-Liu tree approximation

Abstract

This paper considers two mutual-information based approaches for clustering random variables proposed in the literature: clustering by mutual information relevance networks (MIRNs) and clustering by multivariate mutual information (MMI). Despite being two seemingly very different approaches, the derived clustering solutions share very strong structural similarity. Motivated by this curious fact, in this paper we show that there is a precise connection between these two clustering solutions via the celebrated Chow-Liu tree algorithm in machine learning: Under a Chow-Liu tree approximation to the underlying joint distribution, the clustering solutions provided by MIRNs and by MMI are, in fact, identical. This solidifies the heuristic view of clustering by MMI as a natural generalization of clustering by MIRNs from dependency-tree distributions to general joint distributions.