One-to-one lossless codes in the variable input-length regime: Back to Kraft's inequality

Abstract

Unique decodability in the “one-shot” lossless coding scenario, where a single block of source samples is compressed, requires the assignment of distinct codewords to different blocks (one-to-one mapping), without the prefix constraint. As a result, for fixed-length blocks, the corresponding block entropy is not a lower bound on the expected code length, a fact that has recently attracted renewed interest. In this note, we consider an alternative scenario, where the encoder is fed with blocks of arbitrary length, which we argue better reflects the conditions under which one-shot codes may be of any interest. Elaborating on an argument by Rissanen, we first show that the block-entropy is still a fundamental performance bound for one-to-one codes. We then design a code that essentially achieves this bound and satisfies Kraft's inequality for each block length. This code can be implemented with a modification to the termination procedure of the popular Shannon-Fano-Elias code. We conclude that Kraft's inequality is relevant also in the one-shot coding scenario.