Maximum entropy and related methods

Abstract

Originally coming from physics, maximum entropy (ME) has been promoted to a general principle of inference primarily by the works of Jaynes. ME applies to the problem of inferring a probability mass (or density) function, or any non-negative function p(x), when the available information specifies a set E of feasible functions, and there is a prior guess q /spl notin/ E. The author will review the arguments that have been put forward for justifying ME. In this author's opinion, the strongest theoretical support to ME is provided by the axiomatic approach. This shows that, in some sense, ME is the only logically consistent method of inferring a function subject to linear constraints.