An Interpretability Analysis Framework to Enhance Deep Learning Model Transparency: With a Study Case on Flashover Prediction Using Time-Series Sensor Data

Abstract

Deep learning model has been a viable approach to forecast critical events in fire development. However, prior to its implementation in real-life firefighting, it is imperative to further understand the black box and assess its rationale. In this paper, an interpretability analysis framework was proposed to reliably enhance the transparency of deep learning models in time series. The framework was applied to a flashover forecasting model as a case study, including employing an interpretability method to obtain attributions and adapting the evaluation metrics to validate the method’s effectiveness and determine its optimal parameter setting for the model. Results show that the use of the interpretability method, named DeepLIFT, can provide precise attributions to the model inputs in both temporal and spatial domains. Based on the quantitative analysis, suitable parameters were found and the relevance of the attribution results to the model decision was validated, which means the attribution results are reliable to be utilized to interpret the model. It is believed this work would contribute to bringing trustworthy deep learning models for fire research.