Lossy Source Coding via Deep Learning

Abstract

Motivated by a recent work of learning rate distortion approaching posterior via Restricted Boltzmann Machines, we generalize the result to Deep Belief Networks and propose a deep learning based lossy compression for stationary ergodic sources. The compression algorithm consists of two stages, a training stage, which is to learn the posterior with the training data of the same class as the source, and a compression/reproduction stage, which consists of a lossless compression and a lossless reproduction. The theoretical result shows that our algorithm asymptotically achieves the optimum rate-distortion function for stationary ergodic sources, and the experimental results outperform the reported best results.