Constrained Signals: A General Theory of Information Content and Detection

In this paper, a general theory of signals characterized by probabilistic constraints is developed. As in previous work (10), the theoretical development employs Lagrange multipliers to implement the constraints and the maximum en- tropy principle to generate the most likely probability distribution function consistent with the constraints. The method of computing the probability distribution functions is similar to that used in computing partition functions in statistical me- chanics. Simple cases in which exact analytic solutions for the maximum entropy distribution functions and entropy exist are studied and their implications discussed. The application of this technique to the problem of signal detection is ex- plored both theoretically and with simulations. It is demonstrated that the method can readily classify signals governed by different constraint distributions as long as the mean value of the constraints for the two distributions is different. Classi- fying signals governed by the constraint distributions that differ in shape but not in mean value is much more difficult. Some solutions to this problem and extensions of the method are discussed.