Event-based optimization for the continuous-time Markov systems

Abstract

Performance optimization plays an important role in both applied and theoretical research. Recent research provides a unified view, with which the main results in many different areas can be derived or explained using two foundational sensitivities equations. With this approach, event-based optimization has been proposed to overcome the difficulties that the traditional approaches could not solve. However, most of the previous results are on discrete-time Markov systems. In the real world, many practical problems require the model of the continuous-time Markov systems. This paper focuses on extending the event-based optimization approach to the continuous-time Markov systems. As any Markov process can be viewed as a GSMP, we first give a standard description on the GSMP model and then slightly modify it to fit our problem setting. Compared with the event-based optimization with the discrete-time model, in the continuous-time case, in addition to control the probabilities of the controllable events, we need also control the rates of the triggerable events. The final result keeps as intuitive as that for the discrete-time Markov systems, and provides a natural framework for studying the event-based optimization problems.