Information bottleneck with input sampling for attribution

Abstract

In order to facilitate the adoption of deep learning in areas where decisions are of critical importance, understanding the model’s internal workings is paramount. Nevertheless, since most models are considered black boxes, this task is usually not trivial, especially when the user does not have access to the network’s intermediate outputs. In this paper, we propose IBISA, a model-agnostic attribution method that reaches state-of-the-art performance by optimizing sampling masks using the Information Bottleneck Principle. Our method improves on the previously known RISE and IBA techniques by placing the bottleneck right after the image input without complex formulations to estimate the mutual information. The method also requires only twenty forward passes and ten backward passes through the network, which is significantly faster than RISE, which needs at least 4000 forward passes. We evaluated IBISA using a VGG-16 and a ResNET-50 model, showing that our method produces explanations comparable or superior to IBA, RISE, and Grad-CAM but much more efficiently.