Optimal learning of Markov k-tree topology

Abstract

The seminal work of Chow and Liu (1968) shows that approximation of a finite probabilistic system by Markov trees can achieve the minimum information loss with the topology of a maximum spanning tree. Our current paper generalizes the result to Markov networks of tree-width $\le k$, for every fixed $k\ge 2$. In particular, we prove that approximation of a finite probabilistic system with such Markov networks has the minimum information loss when the network topology is achieved with a maximum spanning $k$-tree. While constructing a maximum spanning $k$-tree is intractable for even $k=2$, we show that polynomial algorithms can be ensured by a sufficient condition accommodated by many meaningful applications. In particular, we show an efficient algorithm for learning the optimal topology of higher order correlations among random variables that belong to an underlying linear structure. As an application, we demonstrate effectiveness of this efficient algorithm applied to biomolecular 3D structure prediction.