Centralized and Distributed Lossy Source Coding of Densely Sampled Gaussian Data, with and without Transforms

Abstract

With mean-squared error D as a goal, it is well known that one may approach the rate-distortion function R(D) of a spatially nonbandlimited, time IID, continuous- space, discrete-time Gaussian source by spatially sampling at a sufficiently high rate, applying the Karhunen-Loeve transform to sufficiently long blocks, and independently coding transform coefficients of each type at the first-order rate-distortion function of that type, with a distortion target chosen appropriately for that type. This paper compares and contrasts this classical result with several recently explored alternative schemes for encoding source samples taken at a high rate. The first scheme, which scalar quantizes the samples and then losslessly encodes the quantized samples at their entropy-rate, is known to have rate approaching infinity when distortion is held at D. Is such catastrophic behavior due to the scalar quantizer or to the distributed nature of the quantization? Recent results show that even without a transform, but with distributed vector quantization, it is possible to attain performance that differs from the rate-distortion function by only a finite constant. This suggests it was the scalar quantizer that caused the catastrophic behavior. The final recent result suggests the situation is more nuanced, because it shows that if in the classical scheme scalar quantizers with entropy coding replace the ideal coding of the coefficients at their first-order rate-distortion functions, then again performance differs from the rate-distortion function by a finite constant.