Approximated Newton Algorithm for the Ising Model Inference Speeds Up Convergence, Performs Optimally and Avoids Over-fitting

Abstract

Inverse problems consist in inferring parameters of model distributions that are able to fit properly chosen features of experimental data-sets. The Inverse Ising problem specifically consists of searching for the maximal entropy distribution reproducing frequencies and correlations of a binary data-set. In order to solve this task, we propose an algorithm that takes advantage of the provided by the data knowledge of the log-likelihood function around the solution. We show that the present algorithm is faster than standard gradient ascent methods. Moreover, by looking at the algorithm convergence as a stochastic process, we properly define over-fitting and we show how the present algorithm avoids it by construction.