Mapping fMRI voxel activations to CNN feature space for ease of categorization

Abstract

We observe the impact of using category averaged feature vectors as intermediaries in predicting object categories from fMRI(Functional Magnetic Resonance Imaging) voxel activations. The validation accuracy of state-of-art prediction methods falls drastically when multiple classes are used at the same time, pointing towards the overlapping nature of representations in the voxel activations. To overcome this disadvantage, we map these overlapping representation to a more separable representation. The equivalent of these representations in the field of Computer Vision is a Convolutional Neural Network(CNN) feature vector. After taking into consideration the structural trade-offs the Ventral Temporal Cortex possesses to achieve efficient categorization, we designed a model whose architecture tries to mimic these functional nuances. There are two parts to the implementation - Estimation of feature vectors and efficient category prediction from the estimated feature vectors. We inspected the perceptual similarity of the estimated feature vectors by the use of Annoy tree. We found that Deep ReLU-MLP(Rectified Linear Unit-Multilayer Perceptron) performs better at decoding fMRI voxel activations compared to Sparse Linear Regressor. While inspecting the perceptual neighborhood of the decoded feature vector, we found a significantly higher percentage of the feature vectors predicted from visual perception experiments mapped to the correct neighborhood than in the case of visual imagery experiment.