No-reference image quality assessment based on improved vision transformer and transfer learning

Abstract

To improve the accuracy and generalization performance of the existing no-reference image quality assessment models on small datasets, a no-reference image quality assessment model based on an improved vision transformer model and transfer learning is proposed. Firstly, ResNet is employed as a feature extraction network to obtain basic perceptual features from the input images, and a Convolutional Block Attention Module is introduced to further improve the network's feature extraction capabilities. Secondly, the Transformer Encoder is utilized to regress multi-layer features, improving the network's ability to capture global image information and predict scores. Lastly, to overcome the performance limitations of the Transformer model on small datasets, a transfer learning method is used to solve the dilemma of the relatively small capacity of the databases for image quality assessment. The model is trained and tested on three small-scale datasets and compared with seven mainstream algorithms. Performance is analyzed across three dimensions using statistical significance tests. The results show that, while the model does not perform best in distinguishing between similar and significantly different pairs, it still demonstrates competitive capabilities. Additionally, it performs exceptionally well in assessing quality differences and evaluating Area Under Curve, highlighting its strong potential for practical applications.