New results on third-order coding rate for universal fixed-to-variable source coding

Abstract

Converse and achievable results on the third-order coding rate for universal fixed-to-variable source coding are presented. The authors previously introduced a new universal non-prefix Type Size code for memoryless sources in which codewords lengths are chosen in ascending order of type class sizes. This paper presents a converse on the third-order coding rate for general class of fixed-to-variable codes and proves the optimality of Type Size codes for memoryless sources. The third-order coding rate for a two-stage code is also developed and is shown to be strictly larger than achieved by the Type Size code.