A state-specific joint size, maintenance, and inventory policy for a k-out-of-n load-sharing system subject to self-announcing failures

Abstract

Making optimal decisions for system size and its components’ replacement and spare parts’ replenishment is crucial for the normal operation of a k-out-of-n system whose components deteriorate over time. This task is challenging when the components have load-dependent deterioration rates and the system is subject to self-announcing failures. In this paper, we develop a Semi-Markov decision process (SMDP) based reinforcement learning (RL) framework for designing a joint size and state-specific maintenance and inventory policy for such systems. The proposed policy outperforms the threshold-based policies in minimizing the system's overall cost rate. First, we identify the form of the state-specific policy that determines the number of components to be replaced and replenishment levels at aperiodic decision epochs. Second, we develop an SMDP-based framework for modeling the state transition process and overall cost rate of the system. The structural properties of the optimal policy are obtained, based on which we narrow the search space. Last, we propose a Dueling Double Deep Q-Network (D3QN) algorithm with invalid action masking to alleviate the dimensionality explosion issue caused by the large search space. A case study of a feedwater pump system illustrates the proposed policy's efficiency.