A compositional symbolic calculus approach to producing reduced Markov chains

Abstract

Markov Chains (MCs) are very powerful in capturing the dynamic aspects of systems and in the evaluation of safety measures. However, such models suffer from the state space explosion problem, which often makes their solutions intractable if not impossible. In this paper, a new approach to computing an optimal description of a system MC is presented. The approach is based on an algebraic representation of a Markov chain in a standard sum-of-product canonical form which can then be reduced by symbolic calculus — the sequences are captured by using only the Boolean logic operator AND (symbol ‘.’) and the Priority-OR temporal logic operator (POR, symbol ‘|’). POR is used to represent a priority situation where one event must occur first and other events may or may not occur subsequently. This approach preserves the advantage of using the powerful Boolean methods in the reduction process which is rather extended with temporal logic calculus. By solving the reduced MC, exact measures of interest for the larger MC can be computed. However, since the complete MC needs to be constructed beforehand in order to be reduced afterwards, the approach is practical only via composition. That is, for large systems, a smaller system MC can be produced directly from compositional reduced MCs that are local to the system constituents.