On subspace structure in source and channel coding

The use of subspace structure in source and channel coding is studied. We show that for source coding of an i.i.d. Gaussian source, restriction of the codebook to a union of subspaces need not induce any performance penalty. In fact, in N-dimensional space, a two-stage quantization of first projecting to the nearest of J subspaces of dimension K in a random first-stage codebook of subspaces, followed by quantizing to the nearest of codewords in a second-stage codebook within the K-dimensional subspace induces no performance loss. This structure allows the rate-distortion bound to be approached asymptotically with block length N. The dual results for channel coding are explicitly described: for an additive white Gaussian noise channel, we introduce a particular subspace-based codebook that induces no rate loss, and the Shannon capacity is achieved. While this has complexity exponential in N, it is reduced from an unstructured search.