A computational approach to extreme values and related hitting probabilities in level-dependent quasi-birth–death processes

Abstract

This paper analyzes the dynamics of a level-dependent quasi-birth–death process $X=\{(I\left(t\right),J\left(t\right)):t\ge 0\}$, i.e., a bi-variate Markov chain defined on the countable state space ${\cup }_{i=0}^{\infty }l\left(i\right)$ with $l\left(i\right)=\{(i,j):j\in \{0,\dots ,M_i\}\}$, for integers $M_i\in N_0$ and $i\in N_0$, which has the special property that its $q$-matrix has a block-tridiagonal form. Under the assumption that the first passage to the subset $l\left(0\right)$ occurs in a finite time with certainty, we characterize the probability law of $({\tau }_{max},I_{max},J\left({\tau }_{max}\right))$, where $I_{max}$ is the running maximum level attained by process $X$ before its first visit to states in $l\left(0\right)$, ${\tau }_{max}$ is the first time that the level process $\{I\left(t\right):t\ge 0\}$ reaches the running maximum $I_{max}$, and $J\left({\tau }_{max}\right)$ is the phase at time ${\tau }_{max}$. Our methods rely on the use of restricted Laplace–Stieltjes transforms of ${\tau }_{max}$ on the set of sample paths $\{I_{max}=i,J\left({\tau }_{max}\right)=j\}$, and related processes under taboo of certain subsets of states. The utility of the resulting computational algorithms is demonstrated in two epidemic models: the SIS model for horizontally and vertically transmitted diseases; and the SIR model with constant population size.