A framework for post-prognosis decision-making utilizing deep reinforcement learning considering imperfect maintenance decisions and Value of Information

Abstract

The digitalization era has introduced an abundance of data that can be harnessed to monitor and predict the health of structures. This paper presents a comprehensive framework for post-prognosis decision-making that utilizes deep reinforcement learning (DRL) to manage maintenance decisions on multi-component systems subject to imperfect repairs. The proposed framework integrates raw sensory data acquisition, feature extraction, prognostics, imperfect repair modeling, and decision-making. This integration considers all these tasks independent, promoting flexibility and paving the way for more advanced and adaptable maintenance solutions in real-world applications. The framework’s effectiveness is demonstrated through a case study involving tension-tension fatigue experiments on open-hole aluminum coupons representing multiple dependent components, where the ability to make stochastic RUL estimations and schedule maintenance actions is evaluated. The results demonstrate that the framework can effectively extend the lifecycle of the system while accommodating uncertainties in maintenance actions. This work utilizes the Value of Information to choose the optimal times to acquire new data, resulting in computational efficiency and significant resource savings. Finally, it emphasizes the importance of decomposing uncertainty into epistemic and aleatoric to convert the total uncertainty into decision probabilities over the chosen actions, ensuring reliability and enhancing the interpretability of the DRL model.