A measure of information

Abstract

Modern information theory is founded on the ideas of Hartley and Shannon, amongst others. From a practitioners standpoint. Shannon's probabilistic framework carries certain impediments for the practical measurement of information, such as requiring a priori knowledge of a source's characteristics. Moreover, such a statistical formulation of entropy is an asymptotic limit, meaningful only within the context of an ensemble of messages. It thus fails to address the notion of an individual string having information content in of itself. However Cherry (1953) demonstrated that Shannon's entropy could be viewed equivalently as a measure of the average number of selections required to identify each message symbol from the alphabet. Here the terminology contrasts with Shannon's probabilistic formulation. With the process of counting selection steps appearing to be meaningful for individual, isolated, finite strings. We explore this alternative approach in the context of a recursive hierarchical pattern copying (RHPC) algorithm. We use the measure of the complexity of finite strings, in terms of the number of steps required to recursively construct the string from its alphabet. From this we compute an effective rate of steps-per-symbol required for linearly constructing the string. By Cherry's interpretation of Shannon's entropy we infer this as giving asymptotic equivalence between the two approaches, but perhaps the real significance of this new way to measure information, is its applicability and usefulness in respect of evaluating individual finite strings.