Rate-distortion function for finite block codes: analysis of symmetric binary hamming problem

Abstract

Shannon's rate-distortion theory provides an asymptotic analysis, where delays are allowed to grow unbounded. In practice, real-time applications, such as video streaming and network storage, are subject to certain maximum delay. Accordingly, it is imperative to develop a finite-delay framework for analyzing the rate-distortion limit. In this backdrop, we propose an intuitive generalization of Shannon's asymptotic operational framework to finite block codes. In view of the extreme complexity of such framework, we obtain insight by specializing to the symmetric binary hamming problem. Even upon such specialization, the proposed framework is computationally so intensive that accurate evaluation of the finite-delay rate-distortion function is practical only upto a block length of three. In order to obtain further insight, we then propose a lower-complexity lower bound, based on the partition function of natural numbers, whose computation is practical upto a block length of six. Finally, using a simple combinatorial argument, we propose an upper bound to localize the desired rate-distortion function between our lower and upper bounds.