Statistical structural inference from edge weights using a mixture of gamma distributions

Abstract

Abstract The inference of reliable and meaningful connectivity information from weights representing the affinity between nodes in a graph is an outstanding problem in network science. Usually, this is achieved by simply thresholding the edge weights to distinguish true links from false ones and to obtain a sparse set of connections. Tools developed in statistical mechanics have provided particularly effective ways to locate the optimal threshold so as to preserve the statistical properties of the network structure. Thermodynamic analogies together with statistical mechanical ensembles have been proven to be useful in analysing edge-weighted networks. To extend this work, in this article, we use a statistical mechanical model to describe the probability distribution for edge weights. This models the distribution of edge weights using a mixture of Gamma distributions. Using a two-component Gamma mixture model with components describing the edge and non-edge weight distributions, we use the Expectation–Maximization algorithm to estimate the corresponding Gamma distribution parameters and mixing proportions. This gives the optimal threshold to convert weighted networks to sets of binary-valued connections. Numerical analysis shows that it provides a new way to describe the edge weight probability. Furthermore, using a physical analogy in which the weights are the energies of molecules in a solid, the probability density function for nodes is identical to the degree distribution resulting from a uniform weight on edges. This provides an alternative way to study the degree distribution with the nodal probability function in unweighted networks. We observe a phase transition in the low-temperature region, corresponding to a structural transition caused by applying the threshold. Experimental results on real-world weighted and unweighted networks reveal an improved performance for inferring binary edge connections from edge weights.