Neural Code Converter for Visual Image Representation

Abstract

Brain activity patterns as well as anatomical structure differ from person to person. Although anatomical normalization techniques have been used for functional magnetic resonance imaging studies, there are no standard methods to deal with individual differences in activity patterns. In this study, we propose a method to convert brain activity patterns from one person to another by predicting the intensity of each voxel in one person using a voxel pattern of another. Our "neural code converter" is a statistical model trained by a set of brain activities corresponding to a limited number of random visual images, and can predict brain activity for any unseen visual image. The converter was able to predict the brain activity of one person for unseen visual images, given another person's brain activity. We also confirmed that visual images can be reconstructed from the brain activity predicted from another person. Furthermore, we succeeded in training a classifier using the predicted brain activity, to achieve accurate decoding of measured brain activity. These results suggest that our approach offers a novel tool to compare brain activity patterns across subjects. As predicted brain activity could be used to design a pattern for brain stimulation, the neural code converter may provide a basis for brain-to-brain communication of visual images.