Illustration of the duality between channel coding and rate distortion with side information

Abstract

Digital watermarking can be viewed as channel coding with side information at the encoder (CC-SI); the original data to be watermarked is known to the encoder but not the decoder. Likewise, distributed source coding is rate distortion with side information at the decoder (RD-SI); a noisy observation of the source data to be compressed is available to the decoder but not the encoder. For a Gaussian channel or source, CC-SI and RD-SI are shown to be information-theoretic duals. Ideal coding schemes are presented, and the schemes are interpreted geometrically to highlight dual elements and quantities.