Maximum Likelihood Estimation of Factored Regular Deterministic Stochastic Languages

Abstract

This paper proves that for every class C of stochastic languages defined with the coemission product of finitely many probabilistic, deterministic finite-state acceptors (PDFA) and for every data sequence D of finitely many strings drawn i.i.d. from some stochastic language, the Maximum Likelihood Estimate of D with respect to C can be found efficiently by locally optimizing the parameter values. We show that a consequence of the co-emission product is that each PDFA behaves like an independent factor in a joint distribution. Thus, the likelihood function decomposes in a natural way. We also show that the negative log likelihood function is convex. These results are motivated by the study of Strictly k-Piecewise (SPk) Stochastic Languages, which form a class of stochastic languages which is both linguistically motivated and naturally understood in terms of the coemission product of certain PDFAs.