Probabilistic Monads, Domains and Classical Information

Shannon’s classical information theory [18] uses probabil ity theory to analyze channels as mechanisms for information flow. In this paper, we generalize res ults from [14] for binary channels to show how some more modern tools — probabilistic monads and domain theory in particular — can be used to model classical channels. As initiated in [14], th e point of departure is to consider the family of channels with fixed inputs and outputs, rather than trying to analyze channels one at a time. The results show that domain theory has a role to play in the capacity of channels; in particular, the n× n-stochastic matrices, which are the classical channels hav ing the same sized input as output, admit a quotient compact ordered space which is a domain, and the capacity map factors through this quotient via a Scott-continuous map that measures the quotient domain. We also comment on how some of our results relate to recent discoveries about quantum channels and free affine monoids. Classical information theory has its foundations in the seminal work of Claude Shannon [18], who first conceived of analyzing the behavior of channels using entropy and deriving a formula for channel capacity based on mutual information (cf. [6] for a modern presentation of the basic results). Recent work of Martin, et al. [14] reveals that the theory of compact, affi ne monoids and domain theory can be used to analyze the family of binary channels. In this paper, our goal is to generalize the results in [14] to the case of n× n-channels — channels that have n input ports and n output ports. Our approach also uses the monadic properties of probability distributions to giv e an abstract presentation of how channels arise, and that clarifies the role of the doubly stochastic matrices , which are special channels. While our work focuses on the classical case, the situation around quantum information and quantum channels is also a concern, and we point out how our results relate to some recent work [7, 15] on quantum qubit channels and free affine monoids. While most of the ingredients we piec e together are not new, we believe the approach we present does represent a new way in which to understand families of channels and some of their important features. The rest of the paper is structured as follows. In the next sec tion, we describe three monads based on the probability measures over compact spaces, compact monoids and compact groups. Each of these is used to present some aspect of the classical channels. We then introduce topology, and show how the capacity of a channel can be viewed from a topological perspective. The main result here is that capacity is the maximum distance from the surface determined by the entropy function and the underlying polytope generated by the rows of a channel matrix, viewed as vectors in R n for appropriate n. This leads to a generalization of Jensen’s Lemma that charac terizes strictly concave functions. Domain theory is then introduced, as applied to the finitely-genera ted polytopes residing in a compact convex set, ordered by reverse inclusion. Here we characterize when proper maps measure a domain, in the sense of Martin [13]; a closely related result can be found in [16]. Fi nally, we return to the compact monoid of n× n-stochastic matrices and show that it has a natural, algebra ically-defined pre-order relative to which