Shannon meets Blackwell and Le Cam: Channels, codes, and statistical experiments

Abstract

The Blackwell-Le Cam decision theory provides an approximation framework for statistical experiments in terms of expected risks of optimal decision procedures. The Blackwell partial order formalizes an intuitive notion of which experiment of a given pair is “more informative” for the purposes of inference. The Le Cam deficiency is an approximation measure for any two statistical experiments (with the same parameter space), and it tells us how much we will lose if we base our decisions on one experiment rather than another. In this paper, we develop an extension of the Blackwell-Le Cam theory, starting from a partial ordering for channels introduced by Shannon. In particular, we define a new approximation measure for channels, which we call the Shannon deficiency, and use it to prove an approximation theorem for channel codes that extends an earlier result of Shannon. We also construct a broad class of deficiency-like measures for channels based on generalized divergences, relate them to several alternative notions of capacity, and prove new upper and lower bounds on the Le Cam deficiency.