Quantitative comparison of explainable artificial intelligence methods for nuclear power plant accident diagnosis models

Abstract

The rapid advancement of artificial intelligence (AI) technology based on deep neural networks (DNNs) has spurred active development of DNN-based models in the nuclear domain. Due to the black-box nature of these models and the issue of low explainability, their practical application in safety-critical domains is hindered. To address this, numerous explainable AI (XAI) methods have been proposed. However, the selection of an appropriate XAI method is crucial as its performance significantly depends on various factors; nonetheless, comparative studies of XAI methods are limited within the nuclear domain. This study employs perturbation analysis for the quantitative comparison of XAI methods. A method for selecting an appropriate perturbing value is also proposed based on the concept of information entropy to yield reliable perturbation analysis results. For the experiment, a simple nuclear power plant (NPP) accident diagnosis model was developed to reflect the characteristics of the nuclear domain, and four XAI methods were applied for comparative analysis. The experimental results demonstrate that perturbation analysis and the proposed method are effective for quantitatively comparing the performance of XAI methods.