Rate Distortion Via Restricted Boltzmann Machines

Abstract

Rate distortion is the theoretical foundation of lossy source compression. It addresses the problem of determining the minimal number of bits per symbol that should be communicated so that the source can be approximately reconstructed at the receiver without exceeding a given distortion. Restricted Boltzmann Machines (RBMs)are computational models that have recently been attracting much interest because they can represent any binary sequence distribution.The connections between the above two subjects are that rate distortion is a function of the RBM log partition function, and an RBM can be used to learn the optimal posterior as in the Blahut-Arimoto algorithm. The connection suggests a new tool to learn the optimal posterior and to calculate the N-th order rate distortion function.