Markov Chains, Law of Total Probability, and Recurrence Relations

Abstract

The article describes a general method of estimating average hitting times and the associated probabilities for homogeneous finite Markov chains. Recurrence relations for the estimated values and probabilities are obtained using fundamental formulas of the law of total probability. The described method is applied to three problems of a coupled Markov chain of the VKF method; in each case, the resulting recurrence relations are solvable for exact solutions.