On the performance of vector quantizers empirically designed from dependent sources

Abstract

Suppose we are given n real valued samples Z/sub 1/, Z/sub 2/, ..., Z/sub n/ from a stationary source P. We consider the following question. For a compression scheme that uses blocks of length k, what is the minimal distortion (for encoding the true source P) induced by a vector quantizer of fixed rate R, designed from the training sequence. For a certain class of dependent sources, we derive conditions ensuring that the empirically designed quantizer performs as well (on the average) as the optimal quantizer, for almost every training sequence emitted by the source. In particular, we observe that for a code rate R, the optimal way to choose the dimension of the quantizer is k/sub n/=[(1-/spl delta/)R/sup -1/ log n]. The problem of empirical design of a vector quantizer of fixed dimension k based on a vector valued training sequence X/sub 1/, X/sub 2/, ..., X/sub n/ is also considered. For a class of dependent sources, it is shown that the mean squared error (MSE) of the empirically designed quantizer w.r.t the true source distribution converges to the minimum possible MSE at a rate of O(/spl radic/(log n/n)), for almost every training sequence emitted by the source. In addition, the expected value of the distortion redundancy-the difference between the MSEs of the quantizers-converges to zero for a sequence of increasing block lengths k, if we have at our disposal corresponding training sequences whose length grows as n=2/sup (R+/spl delta/)k/. Some of the derivations extend results in empirical quantizer design using an i.i.d. Training sequence, obtained by Linder et al. (see IEEE Trans. on Info. Theory, vol.40, p.1728-40, 1994) and Merhav and Ziv (see IEEE Trans. on Info. Theory, vol.43, p.1112-23, 1997). Proof of the techniques rely on the results in the theory of empirical processes, indexed by VC function classes.