Statistic Analysis for Probabilistic Processes

Abstract

We associate a statistical vector to a trace and a geometrical embedding to a Markov Decision Process, based on a distance on words, and study basic Membership and Equivalence problems. The Membership problem for a trace \textit{w} and a Markov Decision Process \textit{S} decides if there exists a strategy on \textit{S} which generates with high probability traces close to \textit{w}. We prove that Membership of a trace is \emph{testable} and Equivalence of MDPs is polynomial time approximable. For Probabilistic Automata, Membership is not testable, and approximate Equivalence is undecidable. We give a class of properties, based on results concerning the structure of the tail sigma-field of a finite Markov chain, which characterizes equivalent Markov Decision Processes in this context.