Quantization of binary input DMC at optimal mutual information using constrained shortest path problem

Abstract

We consider the problem of reducing the output alphabet size of a binary input discrete memoryless channel from M to K at minimal loss in mutual information. It was found in [1] that this problem can be solved optimally using a dynamic programming approach, which takes only O(M3) worst-case complexity. We first present a new formulation of the problem, as a K-hop single source shortest path problem (K-hop SSSPP) in a graph G(V,E)with M+1 vertices and(M/2 (M+1)- K/2 (K-1)) edges. This new formulation can in the future serve as a basis to several algorithms on channel quantization. Then we found that the algorithm in [1] has asymptotically optimal complexity in the class of path-comparison based algorithms for general graphs. This implies that we can only expect a constant factor improvement in complexity with any other optimal quantizers, until more specific properties of the graph such as edges and their cost-structure with concave mutual information function are exploited in designing the algorithms (e.g. [2], [3]). We finally present a new optimal quantizer algorithm based on the classic Bellman-Ford algorithm on G, achieving a constant factor improvement in complexity. We claim that our algorithm will be about 50% faster than [1].