Study of Transferability of ImageNet-Based Pretrained Model to Brain Tumor MRI Dataset

Brain tumor detection is an active research problem in the field of computer-aided diagnosis in the medical field. While many works used convolutional neural networks (CNN) with transfer learning and addressed this problem with great performance, the interpretability of these transfer learning models was still unclear. In this paper, four different transfer learning settings were tested over three CNN structures including MobileNet, EfficientNet, and ResNet. The first setting is to use a model without transfer learning, the second setting is to use transfer learning keeping all layers learnable; the third setting is to use transfer learning with the first 1/3 layers frozen; the last setting is to use transfer learning with first 2/3 layers frozen. All the pre-trained models were trained on the ImageNet dataset. For each CNN structure and each transfer learning setting, a model was created and trained on the brain Magnetic Resonance Imaging (MRI) dataset. After all the 12 models had been trained, their performance and learned features were compared. Experimental results indicate that the setting with 1/3 layers frozen outperforms other settings, showing the transferability of the first 1/3 layers of models trained on ImageNet to the brain MRI dataset.