Lightweight Transformer Network and Self-supervised Task for Kinship Verification

Abstract

Kinship verification is one of the interesting and critical problems in computer vision research, with significant progress in the past decades. Meanwhile, Vision Transformer (VIT) has recently achieved impressive success in many domains, including object detection, image recognition, and semantic segmentation, among others. Most of the previous work on kinship verification are based on convolutional or recurrent neural networks. Compared with the local processing of images like convolutions, transformers can effectively understand and process images globally. However, due to overuse, there are many Transformer layers of fully connected layers, and VIT speed is still an issue. Therefore, in this paper, inspired by the recent success of Transformer models in vision tasks, we propose a Transformer-based kinship verification for training and optimizing kinship verification models. We first train the basic vision transformer (VIT-B) with 12 transformer layers, then we reduce the transformer layers to 6 layers, namely VIT-S (Small Vit) and 4 layers, namely VIT-T (Tiny Vit), to make a tradeoff between optimization accuracy and efficiency. As the first attempt to apply Transformer to the kinship verification task, it provides a feasible strategy for kinship research topics and verifies the effectiveness of the method in terms of the accuracy of the experimental results.