Basic Entropy Sets

Abstract

The entropy space for n random variables has dimension 2n -1, one dimension for each joint entropy. When the random variables are known to satisfy given functional dependence relationships, the dimension may be smaller. In this paper, we identify a basis for the restriction of the entropy space to distributions satisfying functional dependence relations specified by an acyclic graph. We provide an efficient algorithm for determination of this basis. One application is a reduction in the computational effort required to compute the linear programming bound for multi-source network coding.