A fast symbol coding scheme with specific application in bulk compression

Abstract

Summary form only given. Given a source alphabet of M symbols and associated probabilities or their estimates, we find a sub-optimal set of codewords using a simple prefix property type iterative algorithm to generate codewords lengths and a look-up table based mapping algorithm for assigning codewords. The expected codeword length L/sub f/ is slightly longer than that obtained for a Huffman code but may also be equal to it. When it is equal, the algorithm generates a larger set of applicable codewords. The time complexity for generating lengths and the associated codewords is less than that with the Huffman code, where these tasks have a complexity of O(M log M), while they are of order O(M) in the new algorithm. For bulk compression, where it is necessary to compress a large number of small files, the algorithm typically shows greater compression efficiency than that obtained with other standard UNIX based compressors.