Grad-CAM based visualization of 3D CNNs in classifying fMRI

Abstract

Deep learning methods have proven promising performance in decoding specific task states based on functional magnetic resonance imaging (fMRI) of the human brain, however, they lack transparency in their decision making, in the sense that it is not straightforward to visualize the features on which the decision was made. In this study, we investigated the decoding of four sensorimotor tasks based on 3D fMRI according to 3D Convolutional Neural Network (3DCNN), and then adopted Grad-CAM algorithms to provide visual explanation from deep networks so as to support the decoding decision.