On Use of the EM Algorithm for Penalized Likelihood Estimation

Abstract

SUMMARY The EM algorithm is a popular approach to maximum likelihood estimation but has not been much used for penalized likelihood or maximum a posteriori estimation. This paper discusses properties of the EM algorithm in such contexts, concentrating on rates of conver- gence, and presents an alternative that is usually more practical and converges at least as quickly. The EM algorithm is a general approach to maximum likelihood estimation, rather than a specific algorithm. Dempster et al. (1977) discussed the method and derived basic properties, demonstrating that a variety of procedures previously developed rather informally could be unified. The common strand to problems where the approach is applicable is a notion of 'incomplete data'; this includes the conventional sense of 'missing data' but is much broader than that. The EM algorithm demon- strates its strength in situations where some hypothetical experiment yields data from which estimation is particularly convenient and economical: the 'incomplete' data actually at hand are regarded as observable functions of these 'complete' data. The resulting algorithms, while usually slow to converge, are often extremely simple and remain practical in large problems where no other approaches may be feasible. Dempster et al. (1977) briefly refer to the use of the same approach to the problem of finding the posterior mode (maximum a posteriori estimate) in a Bayesian estima-