A Feature Structure Based Interpretability Evaluation Approach for Deep Learning

Abstract

The shortcomings of deep learning in interpretability make it difficult to trust such complex black-box models in high-value decision problems. Nowadays, there is still no breakthrough in the research of deep learning interpretability, and people can not see the full picture inside the model. Meanwhile, there is no reliable and universal standard to evaluate the interpretability of deep learning model. Therefore, a deep learning interpretability evaluation method based on the feature structure of deep learning is proposed. Firstly, the trustworthiness evaluation is performed to confirm the robustness of the model with the help of Layer-wise relevance propagation. On this basis, the interpretability of the feature structure is measured based on the relevance between features and outputs. Experiments show that this method can effectively compare the interpretability of models.