General treatment of alphabet-message space and integral representation of entropy

Abstract

In this paper we shall clarify a topological structure of the alphabet-message space of the memory channel in information theory, and study the integral representation of entropy amount from a general view point of a certain generalized message space. In order to apply to the general theory of entropy, the present fashion will develop a message space into more general treatment, in which the basic space X will be assumed to be totally disconnected. As will be shown in §2, the alphabet-message space A is a totally disconnected compact space, and in § 3, a kind of theorem relative to sufficiency for a (/-field generated by a partition and a homeoporphism (cf. Theorem 2) and the others (Theorems 3 and 4) are concerned with the semi-continuity of entropy amount which are general form of Breiman's Theorem [1]. Finally, in §4, it will be discussed about the function h(x) found by Parthasarathy [7] whose integral defines the corresponding amount of entropy (cf. Theorem 5). It is also shown that the results in [9] can be generalized (cf. the footnote 2) below). The function h(x) may give useful and interesting tool for the general theory of entropy of measure preserving automorphism or flow over a probabiltiy space.